2024-01-16T14:43:07.389 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-16T14:43:07.390 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} --archive /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190 --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190/orig.config.yaml 2024-01-16T14:43:07.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-16T14:43:07.510 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7518190' kernel: client: sha1: distro flavor: default kdb: true sha1: 2668949e0971849893a3fb7f177ce9032fc65780 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-clone-stats sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 2668 sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-clone-stats suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJM9PxVisPRU+UgAr+oU+jzlLJ7dCgKZNi1khVAIbHVx8jd9XEJwPTikMbXnSCUlaYnLUNEiPuG8mb21kVYz1U= smithi093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBoC4AG+9lTpJtSCEYm+u6Kx2a7PIsQRMQmaTguTE0kBsS0dDi4YYwYzCH2A/BtKdyJG1YXrZcYHBX/glEVWOA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgress teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-16_14:37:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-16T14:43:07.510 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa; will attempt to use it 2024-01-16T14:43:07.511 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa/tasks 2024-01-16T14:43:07.511 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-16T14:43:07.513 INFO:teuthology.task.internal:Checking packages... 2024-01-16T14:43:07.534 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '73a4532fc4d97adee0b985d4d42874ea351a1c86' 2024-01-16T14:43:07.534 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-16T14:43:07.534 INFO:teuthology.packaging:ref: None 2024-01-16T14:43:07.534 INFO:teuthology.packaging:tag: None 2024-01-16T14:43:07.534 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-01-16T14:43:07.534 INFO:teuthology.packaging:sha1: 73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:43:07.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-clone-stats 2024-01-16T14:43:07.815 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7629.g73a4532f 2024-01-16T14:43:07.816 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-16T14:43:07.824 INFO:teuthology.task.internal:no buildpackages task found 2024-01-16T14:43:07.824 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-16T14:43:07.838 INFO:teuthology.task.internal:Saving configuration 2024-01-16T14:43:07.854 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-16T14:43:07.860 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-16T14:43:07.889 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-16 14:37:59.502104', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJM9PxVisPRU+UgAr+oU+jzlLJ7dCgKZNi1khVAIbHVx8jd9XEJwPTikMbXnSCUlaYnLUNEiPuG8mb21kVYz1U='} 2024-01-16T14:43:07.912 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-16 14:37:59.500709', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBoC4AG+9lTpJtSCEYm+u6Kx2a7PIsQRMQmaTguTE0kBsS0dDi4YYwYzCH2A/BtKdyJG1YXrZcYHBX/glEVWOA='} 2024-01-16T14:43:07.912 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-16T14:43:07.918 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-16T14:43:07.918 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-16T14:43:07.918 INFO:teuthology.run_tasks:Running task console_log... 2024-01-16T14:43:07.985 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe9d9d66160>, signals=[15]) 2024-01-16T14:43:07.985 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-16T14:43:07.992 INFO:teuthology.task.internal:Opening connections... 2024-01-16T14:43:07.992 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:43:07.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:43:08.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:43:08.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:43:08.149 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-16T14:43:08.155 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-01-16T14:43:08.177 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-01-16T14:43:08.177 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-01-16T14:43:08.237 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Red Hat Enterprise Linux" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8.6 (Ootpa)" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:ID="rhel" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="fedora" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8.6" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.redhat.com/" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-16T14:43:08.238 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-16T14:43:08.239 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-01-16T14:43:08.260 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2024-01-16T14:43:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2024-01-16T14:43:08.280 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2024-01-16T14:43:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-16T14:43:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-16T14:43:08.340 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2024-01-16T14:43:08.360 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-16T14:43:08.369 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-16T14:43:08.374 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-16T14:43:08.375 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-01-16T14:43:08.377 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2024-01-16T14:43:08.398 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-16T14:43:08.404 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-16T14:43:08.404 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-01-16T14:43:08.437 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2024-01-16T14:43:08.468 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-16T14:43:08.516 INFO:teuthology.run_tasks:Running task kernel... 2024-01-16T14:43:08.531 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'} 2024-01-16T14:43:08.531 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-01-16T14:43:08.531 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-01-16T14:43:08.531 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-16T14:43:08.532 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-16T14:43:08.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:08.551 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2024-01-16T14:43:08.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:08.552 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-01-16T14:43:08.611 INFO:teuthology.orchestra.run.smithi093.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:08.612 INFO:teuthology.task.kernel:Running kernel on smithi093: 4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:08.612 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum install -y kernel 2024-01-16T14:43:08.614 INFO:teuthology.orchestra.run.smithi026.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:08.614 INFO:teuthology.task.kernel:Running kernel on smithi026: 4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:08.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum install -y kernel 2024-01-16T14:43:10.080 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:10.081 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2024-01-16T14:43:10.213 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:10.213 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:10.213 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:10.248 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:10.248 INFO:teuthology.orchestra.run.smithi026.stdout:Unable to read consumer identity 2024-01-16T14:43:10.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:10.350 INFO:teuthology.orchestra.run.smithi026.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:10.350 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:12.178 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 8.2 kB/s | 3.5 kB 00:00 2024-01-16T14:43:12.268 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2024-01-16T14:43:16.882 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 16 MB 00:04 2024-01-16T14:43:20.454 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 543 kB/s | 24 kB 00:00 2024-01-16T14:43:21.223 INFO:teuthology.orchestra.run.smithi093.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-16T14:43:21.223 INFO:teuthology.orchestra.run.smithi093.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-16T14:43:21.287 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2024-01-16T14:43:21.288 INFO:teuthology.orchestra.run.smithi093.stdout:Nothing to do. 2024-01-16T14:43:21.288 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2024-01-16T14:43:21.448 DEBUG:teuthology.orchestra.run.smithi093:> echo no | sudo yum reinstall kernel || true 2024-01-16T14:43:21.787 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:21.787 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2024-01-16T14:43:21.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:21.839 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:21.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:22.146 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:02 ago on Tue 16 Jan 2024 02:43:20 PM UTC. 2024-01-16T14:43:22.218 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-16T14:43:22.218 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-16T14:43:22.259 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2024-01-16T14:43:22.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum reinstall -y kernel || true 2024-01-16T14:43:22.639 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:22.639 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2024-01-16T14:43:22.696 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:22.696 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:22.696 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:43:23.000 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:03 ago on Tue 16 Jan 2024 02:43:20 PM UTC. 2024-01-16T14:43:23.061 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-16T14:43:23.061 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-16T14:43:23.090 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2024-01-16T14:43:23.136 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q kernel | sort -rV | head -n 1 2024-01-16T14:43:23.322 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:23.323 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:23.323 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-16T14:43:23.323 DEBUG:teuthology.parallel:result is None 2024-01-16T14:43:25.244 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 16 MB 00:12 2024-01-16T14:43:29.225 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 495 kB/s | 24 kB 00:00 2024-01-16T14:43:30.036 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-01-16T14:43:30.036 INFO:teuthology.orchestra.run.smithi026.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-01-16T14:43:30.118 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-01-16T14:43:30.119 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2024-01-16T14:43:30.119 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-01-16T14:43:30.263 DEBUG:teuthology.orchestra.run.smithi026:> echo no | sudo yum reinstall kernel || true 2024-01-16T14:43:30.695 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:30.695 INFO:teuthology.orchestra.run.smithi026.stdout:Unable to read consumer identity 2024-01-16T14:43:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:30.755 INFO:teuthology.orchestra.run.smithi026.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:30.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:31.119 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:02 ago on Tue 16 Jan 2024 02:43:29 PM UTC. 2024-01-16T14:43:31.206 INFO:teuthology.orchestra.run.smithi026.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-16T14:43:31.222 INFO:teuthology.orchestra.run.smithi026.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-16T14:43:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:Error: No packages marked for reinstall. 2024-01-16T14:43:31.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum reinstall -y kernel || true 2024-01-16T14:43:31.713 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2024-01-16T14:43:31.713 INFO:teuthology.orchestra.run.smithi026.stdout:Unable to read consumer identity 2024-01-16T14:43:31.771 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:31.772 INFO:teuthology.orchestra.run.smithi026.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-01-16T14:43:31.772 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:43:32.129 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:00:03 ago on Tue 16 Jan 2024 02:43:29 PM UTC. 2024-01-16T14:43:32.202 INFO:teuthology.orchestra.run.smithi026.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-01-16T14:43:32.202 INFO:teuthology.orchestra.run.smithi026.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-01-16T14:43:32.236 INFO:teuthology.orchestra.run.smithi026.stderr:Error: No packages marked for reinstall. 2024-01-16T14:43:32.293 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q kernel | sort -rV | head -n 1 2024-01-16T14:43:32.467 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:32.467 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-01-16T14:43:32.467 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-01-16T14:43:32.467 DEBUG:teuthology.parallel:result is None 2024-01-16T14:43:32.468 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-16T14:43:32.475 INFO:teuthology.task.internal:Creating test directory... 2024-01-16T14:43:32.475 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-16T14:43:32.477 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-16T14:43:32.498 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-16T14:43:32.505 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-16T14:43:32.512 INFO:teuthology.task.internal:Creating archive directory... 2024-01-16T14:43:32.513 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-16T14:43:32.536 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-16T14:43:32.575 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-16T14:43:32.581 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-16T14:43:32.581 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-16T14:43:32.599 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-16T14:43:32.645 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:43:32.645 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:43:32.663 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:43:32.676 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-16T14:43:32.677 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-16T14:43:32.684 INFO:teuthology.task.internal:Configuring sudo... 2024-01-16T14:43:32.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-16T14:43:32.686 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-16T14:43:32.750 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-16T14:43:32.759 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-16T14:43:32.759 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-16T14:43:32.763 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-16T14:43:32.810 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:43:32.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:43:32.932 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:43:32.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:43:33.035 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:43:33.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-16T14:43:33.108 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:43:33.155 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-16T14:43:33.211 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:43:33.295 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-16T14:43:33.341 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:43:33.341 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-16T14:43:33.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-01-16T14:43:33.412 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2024-01-16T14:43:33.462 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-16T14:43:33.506 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-16T14:43:33.810 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-16T14:43:33.819 INFO:teuthology.task.internal:Starting timer... 2024-01-16T14:43:33.819 INFO:teuthology.run_tasks:Running task pcp... 2024-01-16T14:43:33.870 INFO:teuthology.run_tasks:Running task selinux... 2024-01-16T14:43:33.878 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-16T14:43:33.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-01-16T14:43:33.960 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-01-16T14:43:33.962 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2024-01-16T14:43:34.042 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2024-01-16T14:43:34.044 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-16T14:43:34.044 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-01-16T14:43:34.073 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-01-16T14:43:34.074 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2024-01-16T14:43:34.096 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2024-01-16T14:43:34.096 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi093.front.sepia.ceph.com': 'permissive'} 2024-01-16T14:43:34.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-16T14:43:34.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:34.146 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-16T14:43:34.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:43:34.174 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-16T14:43:34.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-01-16T14:43:34.216 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2024-01-16T14:43:34.245 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-16T14:43:34.254 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-16T14:43:34.259 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-16T14:43:34.381 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-16T14:43:34.397 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-16T14:43:34.398 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi093.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-16T14:51:05.737 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi093.front.sepia.ceph.com')] 2024-01-16T14:51:05.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-16T14:51:05.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:51:05.826 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-01-16T14:51:05.901 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-16T14:51:05.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2024-01-16T14:51:05.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-16T14:51:05.977 DEBUG:teuthology.orchestra.run.smithi093:> true 2024-01-16T14:51:06.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2024-01-16T14:51:06.050 INFO:teuthology.run_tasks:Running task clock... 2024-01-16T14:51:06.060 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-16T14:51:06.060 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-16T14:51:06.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-16T14:51:06.064 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-16T14:51:06.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-16T14:51:06.102 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-16T14:51:06.120 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-16T14:51:06.122 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-16T14:51:06.137 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-16T14:51:06.200 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-01-16T14:51:06.212 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2024-01-16T14:51:06.215 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-01-16T14:51:06.223 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2024-01-16T14:51:06.232 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-16T14:51:06.237 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-16T14:51:06.249 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-16T14:51:06.251 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-16T14:51:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2024-01-16T14:51:06.322 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-16T14:51:06.323 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-01-16T14:51:06.323 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.323 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.323 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.323 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.325 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.328 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-16T14:51:06.329 INFO:teuthology.run_tasks:Running task install... 2024-01-16T14:51:06.338 DEBUG:teuthology.task.install:project ceph 2024-01-16T14:51:06.338 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}} 2024-01-16T14:51:06.338 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'} 2024-01-16T14:51:06.338 INFO:teuthology.task.install:Using flavor: default 2024-01-16T14:51:06.348 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-16T14:51:06.348 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-16T14:51:06.349 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'tag': None, 'wait_for_package': False} 2024-01-16T14:51:06.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:51:06.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'tag': None, 'wait_for_package': False} 2024-01-16T14:51:06.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:51:06.505 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/ 2024-01-16T14:51:06.505 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g73a4532f 2024-01-16T14:51:06.573 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/ 2024-01-16T14:51:06.573 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g73a4532f 2024-01-16T14:51:06.624 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-16T14:51:06.624 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:51:06.624 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-16T14:51:06.654 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-16T14:51:06.655 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/73a4532fc4d97adee0b985d4d42874ea351a1c86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-16T14:51:06.680 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-clone-stats/73a4532fc4d97adee0b985d4d42874ea351a1c86/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-16T14:51:06.681 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:51:06.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-16T14:51:06.718 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-16T14:51:06.718 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/73a4532fc4d97adee0b985d4d42874ea351a1c86/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-16T14:51:06.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-16T14:51:06.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-16T14:51:06.799 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-16T14:51:06.865 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2024-01-16T14:51:06.866 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2024-01-16T14:51:06.882 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-16T14:51:06.914 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-01-16T14:51:06.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-01-16T14:51:07.239 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2024-01-16T14:51:07.351 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2024-01-16T14:51:07.567 INFO:teuthology.orchestra.run.smithi093.stdout:57 files removed 2024-01-16T14:51:07.614 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-16T14:51:07.678 INFO:teuthology.orchestra.run.smithi026.stdout:57 files removed 2024-01-16T14:51:07.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-16T14:51:07.950 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2024-01-16T14:51:08.118 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2024-01-16T14:51:08.863 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 170 kB/s | 81 kB 00:00 2024-01-16T14:51:08.990 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 220 kB/s | 81 kB 00:00 2024-01-16T14:51:09.213 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 30 kB/s | 9.5 kB 00:00 2024-01-16T14:51:09.295 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 36 kB/s | 9.5 kB 00:00 2024-01-16T14:51:09.480 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2024-01-16T14:51:09.683 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2024-01-16T14:51:09.850 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-01-16T14:51:09.889 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 9.0 kB/s | 3.5 kB 00:00 2024-01-16T14:51:11.511 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 16 MB 00:01 2024-01-16T14:51:11.634 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 16 MB 00:01 2024-01-16T14:51:15.199 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 411 kB/s | 24 kB 00:00 2024-01-16T14:51:15.509 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 336 kB/s | 24 kB 00:00 2024-01-16T14:51:15.575 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 29 MB/s | 8.8 MB 00:00 2024-01-16T14:51:16.794 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 65 MB 00:01 2024-01-16T14:51:18.169 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 65 MB 00:01 2024-01-16T14:51:27.427 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 25 MB/s | 8.8 MB 00:00 2024-01-16T14:51:28.418 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 58 MB 00:00 2024-01-16T14:51:30.537 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 58 MB 00:01 2024-01-16T14:51:43.638 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-16T14:51:43.640 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-16T14:51:43.658 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-16T14:51:43.660 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-16T14:51:43.661 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-16T14:51:43.663 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-16T14:51:43.666 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-16T14:51:43.670 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-16T14:51:43.671 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-16T14:51:43.877 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout:=================================================================================================================================== 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout:=================================================================================================================================== 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 6.4 k 2024-01-16T14:51:43.891 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 5.4 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 919 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 171 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 1.6 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 133 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 1.8 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 7.4 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 50 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 14 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 49 M 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 262 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 410 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 263 k 2024-01-16T14:51:43.892 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 18 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 31 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 839 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 139 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 197 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 371 k 2024-01-16T14:51:43.893 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 365 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 107 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 93 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.6 M 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 181 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-16T14:51:43.894 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.8 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.7 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 23 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 28 k 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 2.3 M 2024-01-16T14:51:43.895 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 246 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 4.6 M 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 18 M 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 15 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 24 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 183 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-16T14:51:43.896 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 517 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 6.9 M 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: lua-devel x86_64 5.3.4-12.el8 codeready-builder-for-rhel-8-x86_64-rpms 30 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2024-01-16T14:51:43.897 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 45 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 129 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-16T14:51:43.898 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-16T14:51:43.899 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-16T14:51:43.900 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-16T14:51:43.901 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql 10 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:=================================================================================================================================== 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:Install 131 Packages 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 16 Packages 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:51:43.902 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 218 M 2024-01-16T14:51:43.903 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2024-01-16T14:51:44.859 INFO:teuthology.orchestra.run.smithi093.stdout:(1/147): ceph-base-18.0.0-7629.g73a4532f.el8.x8 8.9 MB/s | 5.4 MB 00:00 2024-01-16T14:51:44.910 INFO:teuthology.orchestra.run.smithi093.stdout:(2/147): ceph-18.0.0-7629.g73a4532f.el8.x86_64. 9.8 kB/s | 6.4 kB 00:00 2024-01-16T14:51:44.977 INFO:teuthology.orchestra.run.smithi093.stdout:(3/147): ceph-immutable-object-cache-18.0.0-762 2.5 MB/s | 171 kB 00:00 2024-01-16T14:51:45.244 INFO:teuthology.orchestra.run.smithi093.stdout:(4/147): ceph-mds-18.0.0-7629.g73a4532f.el8.x86 8.7 MB/s | 2.3 MB 00:00 2024-01-16T14:51:45.552 INFO:teuthology.orchestra.run.smithi093.stdout:(5/147): ceph-mgr-18.0.0-7629.g73a4532f.el8.x86 5.3 MB/s | 1.6 MB 00:00 2024-01-16T14:51:45.577 INFO:teuthology.orchestra.run.smithi093.stdout:(6/147): ceph-fuse-18.0.0-7629.g73a4532f.el8.x8 1.3 MB/s | 919 kB 00:00 2024-01-16T14:51:46.294 INFO:teuthology.orchestra.run.smithi093.stdout:(7/147): ceph-mon-18.0.0-7629.g73a4532f.el8.x86 6.3 MB/s | 4.6 MB 00:00 2024-01-16T14:51:46.545 INFO:teuthology.orchestra.run.smithi093.stdout:(8/147): ceph-common-18.0.0-7629.g73a4532f.el8. 9.8 MB/s | 23 MB 00:02 2024-01-16T14:51:46.611 INFO:teuthology.orchestra.run.smithi093.stdout:(9/147): ceph-selinux-18.0.0-7629.g73a4532f.el8 366 kB/s | 24 kB 00:00 2024-01-16T14:51:47.155 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-16T14:51:47.157 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-01-16T14:51:47.176 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-01-16T14:51:47.178 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-01-16T14:51:47.180 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-01-16T14:51:47.181 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-01-16T14:51:47.184 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-01-16T14:51:47.188 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-01-16T14:51:47.190 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-01-16T14:51:47.421 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-01-16T14:51:47.437 INFO:teuthology.orchestra.run.smithi026.stdout:=================================================================================================================================== 2024-01-16T14:51:47.437 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout:=================================================================================================================================== 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 6.4 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 5.4 M 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 919 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 171 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 1.6 M 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 133 k 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 1.8 M 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 7.4 M 2024-01-16T14:51:47.438 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 50 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 14 M 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 49 M 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 262 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 410 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 263 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 18 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 31 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 839 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 139 k 2024-01-16T14:51:47.439 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 197 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 371 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 365 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 107 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 93 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.6 M 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 181 k 2024-01-16T14:51:47.440 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.8 M 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 3.7 M 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-01-16T14:51:47.441 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 23 M 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 28 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 2.3 M 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 246 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 4.6 M 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 18 M 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g73a4532f.el8 ceph-noarch 15 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 24 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-01-16T14:51:47.442 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 183 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 517 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 6.9 M 2024-01-16T14:51:47.443 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.3.4-12.el8 codeready-builder-for-rhel-8-x86_64-rpms 30 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-01-16T14:51:47.444 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 45 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g73a4532f.el8 ceph 129 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-01-16T14:51:47.445 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-01-16T14:51:47.446 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-01-16T14:51:47.447 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-01-16T14:51:47.448 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql 10 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout:=================================================================================================================================== 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout:Install 131 Packages 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 16 Packages 2024-01-16T14:51:47.449 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:51:47.450 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 218 M 2024-01-16T14:51:47.450 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-01-16T14:51:47.835 INFO:teuthology.orchestra.run.smithi093.stdout:(10/147): ceph-radosgw-18.0.0-7629.g73a4532f.el 9.0 MB/s | 14 MB 00:01 2024-01-16T14:51:47.903 INFO:teuthology.orchestra.run.smithi026.stdout:(1/147): ceph-18.0.0-7629.g73a4532f.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2024-01-16T14:51:47.920 INFO:teuthology.orchestra.run.smithi093.stdout:(11/147): cephfs-mirror-18.0.0-7629.g73a4532f.e 3.0 MB/s | 263 kB 00:00 2024-01-16T14:51:48.012 INFO:teuthology.orchestra.run.smithi093.stdout:(12/147): libcephfs-devel-18.0.0-7629.g73a4532f 338 kB/s | 31 kB 00:00 2024-01-16T14:51:48.287 INFO:teuthology.orchestra.run.smithi093.stdout:(13/147): libcephfs2-18.0.0-7629.g73a4532f.el8. 3.0 MB/s | 839 kB 00:00 2024-01-16T14:51:48.388 INFO:teuthology.orchestra.run.smithi093.stdout:(14/147): libcephsqlite-18.0.0-7629.g73a4532f.e 1.8 MB/s | 183 kB 00:00 2024-01-16T14:51:48.479 INFO:teuthology.orchestra.run.smithi093.stdout:(15/147): librados-devel-18.0.0-7629.g73a4532f. 1.5 MB/s | 139 kB 00:00 2024-01-16T14:51:48.638 INFO:teuthology.orchestra.run.smithi093.stdout:(16/147): libradosstriper1-18.0.0-7629.g73a4532 3.2 MB/s | 517 kB 00:00 2024-01-16T14:51:48.703 INFO:teuthology.orchestra.run.smithi026.stdout:(2/147): ceph-base-18.0.0-7629.g73a4532f.el8.x8 5.2 MB/s | 5.4 MB 00:01 2024-01-16T14:51:48.762 INFO:teuthology.orchestra.run.smithi026.stdout:(3/147): ceph-immutable-object-cache-18.0.0-762 2.9 MB/s | 171 kB 00:00 2024-01-16T14:51:48.821 INFO:teuthology.orchestra.run.smithi093.stdout:(17/147): ceph-osd-18.0.0-7629.g73a4532f.el8.x8 5.6 MB/s | 18 MB 00:03 2024-01-16T14:51:48.913 INFO:teuthology.orchestra.run.smithi093.stdout:(18/147): python3-ceph-argparse-18.0.0-7629.g73 489 kB/s | 45 kB 00:00 2024-01-16T14:51:48.997 INFO:teuthology.orchestra.run.smithi093.stdout:(19/147): python3-ceph-common-18.0.0-7629.g73a4 1.5 MB/s | 129 kB 00:00 2024-01-16T14:51:49.064 INFO:teuthology.orchestra.run.smithi093.stdout:(20/147): python3-cephfs-18.0.0-7629.g73a4532f. 2.9 MB/s | 197 kB 00:00 2024-01-16T14:51:49.079 INFO:teuthology.orchestra.run.smithi026.stdout:(4/147): ceph-mds-18.0.0-7629.g73a4532f.el8.x86 7.3 MB/s | 2.3 MB 00:00 2024-01-16T14:51:49.156 INFO:teuthology.orchestra.run.smithi093.stdout:(21/147): python3-rados-18.0.0-7629.g73a4532f.e 3.9 MB/s | 371 kB 00:00 2024-01-16T14:51:49.248 INFO:teuthology.orchestra.run.smithi093.stdout:(22/147): python3-rbd-18.0.0-7629.g73a4532f.el8 3.9 MB/s | 365 kB 00:00 2024-01-16T14:51:49.332 INFO:teuthology.orchestra.run.smithi093.stdout:(23/147): python3-rgw-18.0.0-7629.g73a4532f.el8 1.2 MB/s | 107 kB 00:00 2024-01-16T14:51:49.346 INFO:teuthology.orchestra.run.smithi026.stdout:(5/147): ceph-mgr-18.0.0-7629.g73a4532f.el8.x86 6.1 MB/s | 1.6 MB 00:00 2024-01-16T14:51:49.398 INFO:teuthology.orchestra.run.smithi093.stdout:(24/147): rbd-fuse-18.0.0-7629.g73a4532f.el8.x8 1.4 MB/s | 93 kB 00:00 2024-01-16T14:51:49.763 INFO:teuthology.orchestra.run.smithi026.stdout:(6/147): ceph-fuse-18.0.0-7629.g73a4532f.el8.x8 494 kB/s | 919 kB 00:01 2024-01-16T14:51:50.032 INFO:teuthology.orchestra.run.smithi093.stdout:(25/147): rbd-mirror-18.0.0-7629.g73a4532f.el8. 5.7 MB/s | 3.6 MB 00:00 2024-01-16T14:51:50.088 INFO:teuthology.orchestra.run.smithi026.stdout:(7/147): ceph-mon-18.0.0-7629.g73a4532f.el8.x86 6.3 MB/s | 4.6 MB 00:00 2024-01-16T14:51:50.090 INFO:teuthology.orchestra.run.smithi093.stdout:(26/147): rbd-nbd-18.0.0-7629.g73a4532f.el8.x86 3.0 MB/s | 181 kB 00:00 2024-01-16T14:51:50.182 INFO:teuthology.orchestra.run.smithi093.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7629.g 305 kB/s | 28 kB 00:00 2024-01-16T14:51:50.266 INFO:teuthology.orchestra.run.smithi093.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7629.g73a4532 1.6 MB/s | 133 kB 00:00 2024-01-16T14:51:50.591 INFO:teuthology.orchestra.run.smithi093.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7629.g73a45 5.4 MB/s | 1.8 MB 00:00 2024-01-16T14:51:50.700 INFO:teuthology.orchestra.run.smithi093.stdout:(30/147): librgw2-18.0.0-7629.g73a4532f.el8.x86 3.4 MB/s | 6.9 MB 00:02 2024-01-16T14:51:50.808 INFO:teuthology.orchestra.run.smithi093.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7629.g73 2.2 MB/s | 246 kB 00:00 2024-01-16T14:51:50.950 INFO:teuthology.orchestra.run.smithi093.stdout:(32/147): ceph-mgr-rook-18.0.0-7629.g73a4532f.e 354 kB/s | 50 kB 00:00 2024-01-16T14:51:51.059 INFO:teuthology.orchestra.run.smithi093.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7629.g7 138 kB/s | 15 kB 00:00 2024-01-16T14:51:51.226 INFO:teuthology.orchestra.run.smithi093.stdout:(34/147): ceph-volume-18.0.0-7629.g73a4532f.el8 1.5 MB/s | 262 kB 00:00 2024-01-16T14:51:51.322 INFO:teuthology.orchestra.run.smithi026.stdout:(8/147): ceph-common-18.0.0-7629.g73a4532f.el8. 6.2 MB/s | 23 MB 00:03 2024-01-16T14:51:51.393 INFO:teuthology.orchestra.run.smithi093.stdout:(35/147): cephadm-18.0.0-7629.g73a4532f.el8.noa 2.4 MB/s | 410 kB 00:00 2024-01-16T14:51:51.438 INFO:teuthology.orchestra.run.smithi026.stdout:(9/147): ceph-selinux-18.0.0-7629.g73a4532f.el8 209 kB/s | 24 kB 00:00 2024-01-16T14:51:51.493 INFO:teuthology.orchestra.run.smithi093.stdout:(36/147): ceph-mgr-diskprediction-local-18.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-01-16T14:51:51.518 INFO:teuthology.orchestra.run.smithi093.stdout:(37/147): cephfs-top-18.0.0-7629.g73a4532f.el8. 145 kB/s | 18 kB 00:00 2024-01-16T14:51:51.719 INFO:teuthology.orchestra.run.smithi093.stdout:(38/147): liboath-2.6.2-3.el8.x86_64.rpm 294 kB/s | 59 kB 00:00 2024-01-16T14:51:51.794 INFO:teuthology.orchestra.run.smithi093.stdout:(39/147): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2024-01-16T14:51:51.811 INFO:teuthology.orchestra.run.smithi093.stdout:(40/147): libunwind-1.3.1-3.el8.x86_64.rpm 819 kB/s | 75 kB 00:00 2024-01-16T14:51:51.870 INFO:teuthology.orchestra.run.smithi093.stdout:(41/147): luarocks-3.5.0-1.el8.noarch.rpm 2.2 MB/s | 167 kB 00:00 2024-01-16T14:51:51.961 INFO:teuthology.orchestra.run.smithi093.stdout:(42/147): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 477 kB/s | 44 kB 00:00 2024-01-16T14:51:51.987 INFO:teuthology.orchestra.run.smithi093.stdout:(43/147): python3-asyncssh-2.7.0-2.el8.noarch.r 2.6 MB/s | 476 kB 00:00 2024-01-16T14:51:52.037 INFO:teuthology.orchestra.run.smithi093.stdout:(44/147): python3-beautifulsoup4-4.6.3-2.el8.1. 2.4 MB/s | 185 kB 00:00 2024-01-16T14:51:52.039 INFO:teuthology.orchestra.run.smithi026.stdout:(10/147): ceph-radosgw-18.0.0-7629.g73a4532f.el 7.1 MB/s | 14 MB 00:01 2024-01-16T14:51:52.054 INFO:teuthology.orchestra.run.smithi093.stdout:(45/147): python3-cachetools-3.1.1-4.el8.noarch 494 kB/s | 33 kB 00:00 2024-01-16T14:51:52.104 INFO:teuthology.orchestra.run.smithi093.stdout:(46/147): python3-certifi-2018.10.15-7.el8.noar 245 kB/s | 16 kB 00:00 2024-01-16T14:51:52.123 INFO:teuthology.orchestra.run.smithi026.stdout:(11/147): cephfs-mirror-18.0.0-7629.g73a4532f.e 3.1 MB/s | 263 kB 00:00 2024-01-16T14:51:52.129 INFO:teuthology.orchestra.run.smithi093.stdout:(47/147): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2024-01-16T14:51:52.181 INFO:teuthology.orchestra.run.smithi026.stdout:(12/147): libcephfs-devel-18.0.0-7629.g73a4532f 534 kB/s | 31 kB 00:00 2024-01-16T14:51:52.298 INFO:teuthology.orchestra.run.smithi026.stdout:(13/147): libcephfs2-18.0.0-7629.g73a4532f.el8. 7.0 MB/s | 839 kB 00:00 2024-01-16T14:51:52.356 INFO:teuthology.orchestra.run.smithi026.stdout:(14/147): libcephsqlite-18.0.0-7629.g73a4532f.e 3.1 MB/s | 183 kB 00:00 2024-01-16T14:51:52.440 INFO:teuthology.orchestra.run.smithi026.stdout:(15/147): librados-devel-18.0.0-7629.g73a4532f. 1.6 MB/s | 139 kB 00:00 2024-01-16T14:51:52.548 INFO:teuthology.orchestra.run.smithi026.stdout:(16/147): libradosstriper1-18.0.0-7629.g73a4532 4.7 MB/s | 517 kB 00:00 2024-01-16T14:51:52.563 INFO:teuthology.orchestra.run.smithi093.stdout:(48/147): ceph-test-18.0.0-7629.g73a4532f.el8.x 8.3 MB/s | 49 MB 00:05 2024-01-16T14:51:52.588 INFO:teuthology.orchestra.run.smithi093.stdout:(49/147): python3-google-auth-1.1.1-10.el8.noar 177 kB/s | 81 kB 00:00 2024-01-16T14:51:52.613 INFO:teuthology.orchestra.run.smithi093.stdout:(50/147): python3-cherrypy-18.4.0-1.el8.noarch. 756 kB/s | 384 kB 00:00 2024-01-16T14:51:52.655 INFO:teuthology.orchestra.run.smithi093.stdout:(51/147): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2024-01-16T14:51:52.697 INFO:teuthology.orchestra.run.smithi093.stdout:(52/147): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2024-01-16T14:51:52.723 INFO:teuthology.orchestra.run.smithi093.stdout:(53/147): python3-logutils-0.3.5-11.el8.noarch. 729 kB/s | 49 kB 00:00 2024-01-16T14:51:52.773 INFO:teuthology.orchestra.run.smithi093.stdout:(54/147): python3-kubernetes-11.0.0-6.el8.noarc 7.3 MB/s | 1.2 MB 00:00 2024-01-16T14:51:52.823 INFO:teuthology.orchestra.run.smithi093.stdout:(55/147): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-01-16T14:51:52.848 INFO:teuthology.orchestra.run.smithi093.stdout:(56/147): python3-more-itertools-7.2.0-3.el8.no 390 kB/s | 59 kB 00:00 2024-01-16T14:51:52.924 INFO:teuthology.orchestra.run.smithi093.stdout:(57/147): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2024-01-16T14:51:52.966 INFO:teuthology.orchestra.run.smithi093.stdout:(58/147): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-01-16T14:51:52.991 INFO:teuthology.orchestra.run.smithi093.stdout:(59/147): python3-repoze-lru-0.7-6.el8.noarch.r 237 kB/s | 34 kB 00:00 2024-01-16T14:51:53.016 INFO:teuthology.orchestra.run.smithi093.stdout:(60/147): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2024-01-16T14:51:53.065 INFO:teuthology.orchestra.run.smithi093.stdout:(61/147): python3-rsa-4.9-2.el8.noarch.rpm 649 kB/s | 64 kB 00:00 2024-01-16T14:51:53.116 INFO:teuthology.orchestra.run.smithi026.stdout:(17/147): librgw2-18.0.0-7629.g73a4532f.el8.x86 12 MB/s | 6.9 MB 00:00 2024-01-16T14:51:53.159 INFO:teuthology.orchestra.run.smithi093.stdout:(62/147): python3-simplegeneric-0.8.1-17.el8.no 112 kB/s | 19 kB 00:00 2024-01-16T14:51:53.165 INFO:teuthology.orchestra.run.smithi026.stdout:(18/147): python3-ceph-argparse-18.0.0-7629.g73 903 kB/s | 45 kB 00:00 2024-01-16T14:51:53.175 INFO:teuthology.orchestra.run.smithi093.stdout:(63/147): python3-singledispatch-3.4.0.3-18.el8 151 kB/s | 24 kB 00:00 2024-01-16T14:51:53.192 INFO:teuthology.orchestra.run.smithi093.stdout:(64/147): python3-tempora-1.14.1-5.el8.noarch.r 230 kB/s | 29 kB 00:00 2024-01-16T14:51:53.225 INFO:teuthology.orchestra.run.smithi093.stdout:(65/147): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2024-01-16T14:51:53.256 INFO:teuthology.orchestra.run.smithi026.stdout:(19/147): python3-ceph-common-18.0.0-7629.g73a4 1.4 MB/s | 129 kB 00:00 2024-01-16T14:51:53.276 INFO:teuthology.orchestra.run.smithi093.stdout:(66/147): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2024-01-16T14:51:53.301 INFO:teuthology.orchestra.run.smithi093.stdout:(67/147): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-01-16T14:51:53.308 INFO:teuthology.orchestra.run.smithi026.stdout:(20/147): python3-cephfs-18.0.0-7629.g73a4532f. 3.8 MB/s | 197 kB 00:00 2024-01-16T14:51:53.360 INFO:teuthology.orchestra.run.smithi093.stdout:(68/147): python3-websocket-client-0.56.0-5.el8 457 kB/s | 61 kB 00:00 2024-01-16T14:51:53.375 INFO:teuthology.orchestra.run.smithi026.stdout:(21/147): python3-rados-18.0.0-7629.g73a4532f.e 5.5 MB/s | 371 kB 00:00 2024-01-16T14:51:53.385 INFO:teuthology.orchestra.run.smithi093.stdout:(69/147): python3-webtest-2.0.33-1.el8.noarch.r 786 kB/s | 85 kB 00:00 2024-01-16T14:51:53.410 INFO:teuthology.orchestra.run.smithi093.stdout:(70/147): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-01-16T14:51:53.436 INFO:teuthology.orchestra.run.smithi093.stdout:(71/147): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-01-16T14:51:53.442 INFO:teuthology.orchestra.run.smithi026.stdout:(22/147): python3-rbd-18.0.0-7629.g73a4532f.el8 5.3 MB/s | 365 kB 00:00 2024-01-16T14:51:53.494 INFO:teuthology.orchestra.run.smithi093.stdout:(72/147): lua-devel-5.3.4-12.el8.x86_64.rpm 359 kB/s | 30 kB 00:00 2024-01-16T14:51:53.500 INFO:teuthology.orchestra.run.smithi026.stdout:(23/147): python3-rgw-18.0.0-7629.g73a4532f.el8 1.8 MB/s | 107 kB 00:00 2024-01-16T14:51:53.519 INFO:teuthology.orchestra.run.smithi093.stdout:(73/147): libverto-devel-0.3.0-5.el8.x86_64.rpm 229 kB/s | 18 kB 00:00 2024-01-16T14:51:53.544 INFO:teuthology.orchestra.run.smithi093.stdout:(74/147): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2024-01-16T14:51:53.551 INFO:teuthology.orchestra.run.smithi026.stdout:(24/147): rbd-fuse-18.0.0-7629.g73a4532f.el8.x8 1.8 MB/s | 93 kB 00:00 2024-01-16T14:51:53.595 INFO:teuthology.orchestra.run.smithi093.stdout:(75/147): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2024-01-16T14:51:53.628 INFO:teuthology.orchestra.run.smithi093.stdout:(76/147): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2024-01-16T14:51:53.654 INFO:teuthology.orchestra.run.smithi093.stdout:(77/147): python3-requests-oauthlib-1.0.0-1.el8 726 kB/s | 43 kB 00:00 2024-01-16T14:51:53.679 INFO:teuthology.orchestra.run.smithi093.stdout:(78/147): libaio-devel-0.3.112-1.el8.x86_64.rpm 374 kB/s | 19 kB 00:00 2024-01-16T14:51:53.704 INFO:teuthology.orchestra.run.smithi093.stdout:(79/147): libzstd-devel-1.4.4-1.el8.x86_64.rpm 872 kB/s | 44 kB 00:00 2024-01-16T14:51:53.738 INFO:teuthology.orchestra.run.smithi093.stdout:(80/147): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2024-01-16T14:51:53.763 INFO:teuthology.orchestra.run.smithi093.stdout:(81/147): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2024-01-16T14:51:53.788 INFO:teuthology.orchestra.run.smithi093.stdout:(82/147): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-01-16T14:51:53.814 INFO:teuthology.orchestra.run.smithi093.stdout:(83/147): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2024-01-16T14:51:53.867 INFO:teuthology.orchestra.run.smithi026.stdout:(25/147): rbd-mirror-18.0.0-7629.g73a4532f.el8. 11 MB/s | 3.6 MB 00:00 2024-01-16T14:51:53.868 INFO:teuthology.orchestra.run.smithi093.stdout:(84/147): keyutils-libs-devel-1.5.10-9.el8.x86_ 605 kB/s | 48 kB 00:00 2024-01-16T14:51:53.898 INFO:teuthology.orchestra.run.smithi093.stdout:(85/147): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-01-16T14:51:53.910 INFO:teuthology.orchestra.run.smithi026.stdout:(26/147): rbd-nbd-18.0.0-7629.g73a4532f.el8.x86 4.3 MB/s | 181 kB 00:00 2024-01-16T14:51:53.923 INFO:teuthology.orchestra.run.smithi093.stdout:(86/147): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-01-16T14:51:53.948 INFO:teuthology.orchestra.run.smithi093.stdout:(87/147): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2024-01-16T14:51:53.979 INFO:teuthology.orchestra.run.smithi026.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7629.g 403 kB/s | 28 kB 00:00 2024-01-16T14:51:53.990 INFO:teuthology.orchestra.run.smithi093.stdout:(88/147): thrift-0.13.0-2.el8.x86_64.rpm 2.8 MB/s | 1.7 MB 00:00 2024-01-16T14:51:54.015 INFO:teuthology.orchestra.run.smithi093.stdout:(89/147): fuse-2.9.7-17.el8.x86_64.rpm 902 kB/s | 83 kB 00:00 2024-01-16T14:51:54.057 INFO:teuthology.orchestra.run.smithi093.stdout:(90/147): libgfortran-8.5.0-20.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2024-01-16T14:51:54.083 INFO:teuthology.orchestra.run.smithi093.stdout:(91/147): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.6 MB/s | 100 kB 00:00 2024-01-16T14:51:54.085 INFO:teuthology.orchestra.run.smithi026.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7629.g73a4532 1.2 MB/s | 133 kB 00:00 2024-01-16T14:51:54.138 INFO:teuthology.orchestra.run.smithi093.stdout:(92/147): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-01-16T14:51:54.175 INFO:teuthology.orchestra.run.smithi093.stdout:(93/147): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2024-01-16T14:51:54.200 INFO:teuthology.orchestra.run.smithi093.stdout:(94/147): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 1.1 MB/s | 73 kB 00:00 2024-01-16T14:51:54.225 INFO:teuthology.orchestra.run.smithi093.stdout:(95/147): elfutils-libelf-devel-0.189-3.el8.x86 1.2 MB/s | 62 kB 00:00 2024-01-16T14:51:54.251 INFO:teuthology.orchestra.run.smithi093.stdout:(96/147): librabbitmq-0.9.0-4.el8.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-01-16T14:51:54.260 INFO:teuthology.orchestra.run.smithi026.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7629.g73a45 10 MB/s | 1.8 MB 00:00 2024-01-16T14:51:54.284 INFO:teuthology.orchestra.run.smithi093.stdout:(97/147): python3-cryptography-3.2.1-6.el8.x86_ 2.7 MB/s | 559 kB 00:00 2024-01-16T14:51:54.318 INFO:teuthology.orchestra.run.smithi093.stdout:(98/147): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 6.0 MB/s | 562 kB 00:00 2024-01-16T14:51:54.343 INFO:teuthology.orchestra.run.smithi093.stdout:(99/147): NetworkManager-initscripts-updown-1.4 2.4 MB/s | 144 kB 00:00 2024-01-16T14:51:54.368 INFO:teuthology.orchestra.run.smithi093.stdout:(100/147): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2024-01-16T14:51:54.419 INFO:teuthology.orchestra.run.smithi093.stdout:(101/147): python3-prettytable-0.7.2-14.el8.noa 584 kB/s | 44 kB 00:00 2024-01-16T14:51:54.444 INFO:teuthology.orchestra.run.smithi093.stdout:(102/147): python3-pyasn1-modules-0.3.7-6.el8.n 1.4 MB/s | 110 kB 00:00 2024-01-16T14:51:54.469 INFO:teuthology.orchestra.run.smithi093.stdout:(103/147): python3-jsonpatch-1.21-2.el8.noarch. 531 kB/s | 27 kB 00:00 2024-01-16T14:51:54.494 INFO:teuthology.orchestra.run.smithi093.stdout:(104/147): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2024-01-16T14:51:54.528 INFO:teuthology.orchestra.run.smithi093.stdout:(105/147): python3-werkzeug-0.12.2-4.el8.noarch 7.6 MB/s | 457 kB 00:00 2024-01-16T14:51:54.561 INFO:teuthology.orchestra.run.smithi093.stdout:(106/147): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2024-01-16T14:51:54.595 INFO:teuthology.orchestra.run.smithi093.stdout:(107/147): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2024-01-16T14:51:54.637 INFO:teuthology.orchestra.run.smithi093.stdout:(108/147): openssl-devel-1.1.1k-12.el8_9.x86_64 6.0 MB/s | 2.3 MB 00:00 2024-01-16T14:51:54.679 INFO:teuthology.orchestra.run.smithi093.stdout:(109/147): python3-markupsafe-0.23-19.el8.x86_6 464 kB/s | 39 kB 00:00 2024-01-16T14:51:54.704 INFO:teuthology.orchestra.run.smithi093.stdout:(110/147): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2024-01-16T14:51:54.721 INFO:teuthology.orchestra.run.smithi093.stdout:(111/147): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-01-16T14:51:54.755 INFO:teuthology.orchestra.run.smithi093.stdout:(112/147): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2024-01-16T14:51:54.772 INFO:teuthology.orchestra.run.smithi093.stdout:(113/147): uuid-1.6.2-43.el8.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-01-16T14:51:54.797 INFO:teuthology.orchestra.run.smithi093.stdout:(114/147): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.3 MB/s | 103 kB 00:00 2024-01-16T14:51:54.839 INFO:teuthology.orchestra.run.smithi093.stdout:(115/147): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2024-01-16T14:51:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:(30/147): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2024-01-16T14:51:54.897 INFO:teuthology.orchestra.run.smithi093.stdout:(116/147): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-01-16T14:51:54.936 INFO:teuthology.orchestra.run.smithi026.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7629.g73 4.1 MB/s | 246 kB 00:00 2024-01-16T14:51:54.948 INFO:teuthology.orchestra.run.smithi093.stdout:(117/147): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2024-01-16T14:51:54.978 INFO:teuthology.orchestra.run.smithi026.stdout:(32/147): ceph-mgr-rook-18.0.0-7629.g73a4532f.e 1.2 MB/s | 50 kB 00:00 2024-01-16T14:51:55.006 INFO:teuthology.orchestra.run.smithi093.stdout:(118/147): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2024-01-16T14:51:55.011 INFO:teuthology.orchestra.run.smithi026.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7629.g7 451 kB/s | 15 kB 00:00 2024-01-16T14:51:55.040 INFO:teuthology.orchestra.run.smithi093.stdout:(119/147): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2024-01-16T14:51:55.087 INFO:teuthology.orchestra.run.smithi026.stdout:(34/147): ceph-volume-18.0.0-7629.g73a4532f.el8 3.4 MB/s | 262 kB 00:00 2024-01-16T14:51:55.107 INFO:teuthology.orchestra.run.smithi093.stdout:(120/147): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2024-01-16T14:51:55.132 INFO:teuthology.orchestra.run.smithi093.stdout:(121/147): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-01-16T14:51:55.145 INFO:teuthology.orchestra.run.smithi026.stdout:(35/147): cephadm-18.0.0-7629.g73a4532f.el8.noa 6.9 MB/s | 410 kB 00:00 2024-01-16T14:51:55.196 INFO:teuthology.orchestra.run.smithi026.stdout:(36/147): cephfs-top-18.0.0-7629.g73a4532f.el8. 364 kB/s | 18 kB 00:00 2024-01-16T14:51:55.216 INFO:teuthology.orchestra.run.smithi093.stdout:(122/147): automake-1.16.1-8.el8.noarch.rpm 8.4 MB/s | 714 kB 00:00 2024-01-16T14:51:55.266 INFO:teuthology.orchestra.run.smithi093.stdout:(123/147): libpq-13.11-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2024-01-16T14:51:55.342 INFO:teuthology.orchestra.run.smithi093.stdout:(124/147): openblas-threads-0.3.15-6.el8.x86_64 16 MB/s | 5.2 MB 00:00 2024-01-16T14:51:55.408 INFO:teuthology.orchestra.run.smithi093.stdout:(125/147): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-01-16T14:51:55.446 INFO:teuthology.orchestra.run.smithi026.stdout:(37/147): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-01-16T14:51:55.452 INFO:teuthology.orchestra.run.smithi093.stdout:(126/147): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2024-01-16T14:51:55.484 INFO:teuthology.orchestra.run.smithi093.stdout:(127/147): python3-pytz-2017.2-11.el8.noarch.rp 382 kB/s | 54 kB 00:00 2024-01-16T14:51:55.517 INFO:teuthology.orchestra.run.smithi093.stdout:(128/147): postgresql-contrib-10.23-3.module+el 12 MB/s | 811 kB 00:00 2024-01-16T14:51:55.559 INFO:teuthology.orchestra.run.smithi093.stdout:(129/147): postgresql-10.23-3.module+el8.9.0+20 10 MB/s | 1.5 MB 00:00 2024-01-16T14:51:55.668 INFO:teuthology.orchestra.run.smithi093.stdout:(130/147): postgresql-server-10.23-3.module+el8 27 MB/s | 5.1 MB 00:00 2024-01-16T14:51:55.876 INFO:teuthology.orchestra.run.smithi093.stdout:(131/147): python3-scipy-1.0.0-21.module+el8.9. 38 MB/s | 14 MB 00:00 2024-01-16T14:51:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:(132/147): elfutils-libelf-0.189-3.el8.x86_64.r 4.6 MB/s | 232 kB 00:00 2024-01-16T14:51:55.938 INFO:teuthology.orchestra.run.smithi026.stdout:(38/147): ceph-test-18.0.0-7629.g73a4532f.el8.x 11 MB/s | 49 MB 00:04 2024-01-16T14:51:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:(39/147): liboath-2.6.2-3.el8.x86_64.rpm 114 kB/s | 59 kB 00:00 2024-01-16T14:51:56.002 INFO:teuthology.orchestra.run.smithi093.stdout:(133/147): elfutils-libs-0.189-3.el8.x86_64.rpm 3.9 MB/s | 303 kB 00:00 2024-01-16T14:51:56.047 INFO:teuthology.orchestra.run.smithi026.stdout:(40/147): luarocks-3.5.0-1.el8.noarch.rpm 2.0 MB/s | 167 kB 00:00 2024-01-16T14:51:56.052 INFO:teuthology.orchestra.run.smithi093.stdout:(134/147): libmount-2.32.1-43.el8.x86_64.rpm 4.6 MB/s | 237 kB 00:00 2024-01-16T14:51:56.094 INFO:teuthology.orchestra.run.smithi093.stdout:(135/147): libsmartcols-2.32.1-43.el8.x86_64.rp 4.2 MB/s | 179 kB 00:00 2024-01-16T14:51:56.097 INFO:teuthology.orchestra.run.smithi026.stdout:(41/147): libunwind-1.3.1-3.el8.x86_64.rpm 473 kB/s | 75 kB 00:00 2024-01-16T14:51:56.148 INFO:teuthology.orchestra.run.smithi026.stdout:(42/147): python3-asyncssh-2.7.0-2.el8.noarch.r 4.6 MB/s | 476 kB 00:00 2024-01-16T14:51:56.152 INFO:teuthology.orchestra.run.smithi093.stdout:(136/147): librbd1-18.0.0-7629.g73a4532f.el8.x8 7.7 MB/s | 3.7 MB 00:00 2024-01-16T14:51:56.177 INFO:teuthology.orchestra.run.smithi093.stdout:(137/147): libuuid-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 99 kB 00:00 2024-01-16T14:51:56.181 INFO:teuthology.orchestra.run.smithi026.stdout:(43/147): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2024-01-16T14:51:56.198 INFO:teuthology.orchestra.run.smithi026.stdout:(44/147): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2024-01-16T14:51:56.270 INFO:teuthology.orchestra.run.smithi093.stdout:(138/147): util-linux-2.32.1-43.el8.x86_64.rpm 21 MB/s | 2.5 MB 00:00 2024-01-16T14:51:56.290 INFO:teuthology.orchestra.run.smithi026.stdout:(45/147): python3-certifi-2018.10.15-7.el8.noar 177 kB/s | 16 kB 00:00 2024-01-16T14:51:56.295 INFO:teuthology.orchestra.run.smithi093.stdout:(139/147): util-linux-user-2.32.1-43.el8.x86_64 879 kB/s | 103 kB 00:00 2024-01-16T14:51:56.315 INFO:teuthology.orchestra.run.smithi026.stdout:(46/147): python3-cachetools-3.1.1-4.el8.noarch 246 kB/s | 33 kB 00:00 2024-01-16T14:51:56.328 INFO:teuthology.orchestra.run.smithi093.stdout:(140/147): iproute-6.2.0-5.el8_9.x86_64.rpm 14 MB/s | 854 kB 00:00 2024-01-16T14:51:56.353 INFO:teuthology.orchestra.run.smithi093.stdout:(141/147): libblkid-2.32.1-43.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2024-01-16T14:51:56.371 INFO:teuthology.orchestra.run.smithi093.stdout:(142/147): libfdisk-2.32.1-43.el8.x86_64.rpm 5.9 MB/s | 254 kB 00:00 2024-01-16T14:51:56.374 INFO:teuthology.orchestra.run.smithi026.stdout:(47/147): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-01-16T14:51:56.413 INFO:teuthology.orchestra.run.smithi093.stdout:(143/147): xfsprogs-5.0.0-12.el8.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-01-16T14:51:56.454 INFO:teuthology.orchestra.run.smithi093.stdout:(144/147): openssl-1.1.1k-12.el8_9.x86_64.rpm 8.3 MB/s | 711 kB 00:00 2024-01-16T14:51:56.458 INFO:teuthology.orchestra.run.smithi026.stdout:(48/147): python3-google-auth-1.1.1-10.el8.noar 975 kB/s | 81 kB 00:00 2024-01-16T14:51:56.491 INFO:teuthology.orchestra.run.smithi026.stdout:(49/147): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2024-01-16T14:51:56.525 INFO:teuthology.orchestra.run.smithi026.stdout:(50/147): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-01-16T14:51:56.530 INFO:teuthology.orchestra.run.smithi093.stdout:(145/147): librados2-18.0.0-7629.g73a4532f.el8. 3.9 MB/s | 3.8 MB 00:00 2024-01-16T14:51:56.555 INFO:teuthology.orchestra.run.smithi093.stdout:(146/147): xfsdump-3.1.8-7.el8_9.x86_64.rpm 3.2 MB/s | 333 kB 00:00 2024-01-16T14:51:56.558 INFO:teuthology.orchestra.run.smithi026.stdout:(51/147): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2024-01-16T14:51:56.588 INFO:teuthology.orchestra.run.smithi093.stdout:(147/147): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-01-16T14:51:56.591 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:51:56.591 INFO:teuthology.orchestra.run.smithi093.stdout:Total 17 MB/s | 218 MB 00:12 2024-01-16T14:51:56.646 INFO:teuthology.orchestra.run.smithi026.stdout:(52/147): python3-logutils-0.3.5-11.el8.noarch. 559 kB/s | 49 kB 00:00 2024-01-16T14:51:56.685 INFO:teuthology.orchestra.run.smithi026.stdout:(53/147): python3-kubernetes-11.0.0-6.el8.noarc 7.3 MB/s | 1.2 MB 00:00 2024-01-16T14:51:56.726 INFO:teuthology.orchestra.run.smithi026.stdout:(54/147): python3-more-itertools-7.2.0-3.el8.no 734 kB/s | 59 kB 00:00 2024-01-16T14:51:56.751 INFO:teuthology.orchestra.run.smithi026.stdout:(55/147): python3-natsort-7.1.1-2.el8.noarch.rp 901 kB/s | 60 kB 00:00 2024-01-16T14:51:56.827 INFO:teuthology.orchestra.run.smithi026.stdout:(56/147): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2024-01-16T14:51:56.852 INFO:teuthology.orchestra.run.smithi026.stdout:(57/147): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2024-01-16T14:51:56.902 INFO:teuthology.orchestra.run.smithi026.stdout:(58/147): python3-repoze-lru-0.7-6.el8.noarch.r 447 kB/s | 34 kB 00:00 2024-01-16T14:51:56.953 INFO:teuthology.orchestra.run.smithi026.stdout:(59/147): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2024-01-16T14:51:56.995 INFO:teuthology.orchestra.run.smithi026.stdout:(60/147): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2024-01-16T14:51:57.054 INFO:teuthology.orchestra.run.smithi026.stdout:(61/147): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2024-01-16T14:51:57.112 INFO:teuthology.orchestra.run.smithi026.stdout:(62/147): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2024-01-16T14:51:57.138 INFO:teuthology.orchestra.run.smithi026.stdout:(63/147): python3-tempora-1.14.1-5.el8.noarch.r 350 kB/s | 29 kB 00:00 2024-01-16T14:51:57.213 INFO:teuthology.orchestra.run.smithi026.stdout:(64/147): python3-trustme-0.6.0-4.el8.noarch.rp 275 kB/s | 27 kB 00:00 2024-01-16T14:51:57.238 INFO:teuthology.orchestra.run.smithi026.stdout:(65/147): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2024-01-16T14:51:57.296 INFO:teuthology.orchestra.run.smithi026.stdout:(66/147): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-01-16T14:51:57.322 INFO:teuthology.orchestra.run.smithi026.stdout:(67/147): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-01-16T14:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout:(68/147): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2024-01-16T14:51:57.439 INFO:teuthology.orchestra.run.smithi026.stdout:(69/147): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2024-01-16T14:51:57.481 INFO:teuthology.orchestra.run.smithi026.stdout:(70/147): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2024-01-16T14:51:57.506 INFO:teuthology.orchestra.run.smithi026.stdout:(71/147): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2024-01-16T14:51:57.557 INFO:teuthology.orchestra.run.smithi026.stdout:(72/147): libverto-devel-0.3.0-5.el8.x86_64.rpm 238 kB/s | 18 kB 00:00 2024-01-16T14:51:57.574 INFO:teuthology.orchestra.run.smithi026.stdout:(73/147): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2024-01-16T14:51:57.599 INFO:teuthology.orchestra.run.smithi026.stdout:(74/147): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2024-01-16T14:51:57.605 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2024-01-16T14:51:57.624 INFO:teuthology.orchestra.run.smithi026.stdout:(75/147): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2024-01-16T14:51:57.650 INFO:teuthology.orchestra.run.smithi026.stdout:(76/147): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2024-01-16T14:51:57.675 INFO:teuthology.orchestra.run.smithi026.stdout:(77/147): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2024-01-16T14:51:57.700 INFO:teuthology.orchestra.run.smithi026.stdout:(78/147): libzstd-devel-1.4.4-1.el8.x86_64.rpm 874 kB/s | 44 kB 00:00 2024-01-16T14:51:57.763 INFO:teuthology.orchestra.run.smithi026.stdout:(79/147): pcre2-devel-10.32-2.el8.x86_64.rpm 6.7 MB/s | 605 kB 00:00 2024-01-16T14:51:57.793 INFO:teuthology.orchestra.run.smithi026.stdout:(80/147): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2024-01-16T14:51:57.809 INFO:teuthology.orchestra.run.smithi026.stdout:(81/147): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.7 MB/s | 220 kB 00:00 2024-01-16T14:51:57.835 INFO:teuthology.orchestra.run.smithi026.stdout:(82/147): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2024-01-16T14:51:57.860 INFO:teuthology.orchestra.run.smithi026.stdout:(83/147): keyutils-libs-devel-1.5.10-9.el8.x86_ 957 kB/s | 48 kB 00:00 2024-01-16T14:51:57.885 INFO:teuthology.orchestra.run.smithi026.stdout:(84/147): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-01-16T14:51:57.911 INFO:teuthology.orchestra.run.smithi026.stdout:(85/147): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-01-16T14:51:57.936 INFO:teuthology.orchestra.run.smithi026.stdout:(86/147): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2024-01-16T14:51:57.952 INFO:teuthology.orchestra.run.smithi026.stdout:(87/147): fuse-2.9.7-17.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2024-01-16T14:51:57.995 INFO:teuthology.orchestra.run.smithi026.stdout:(88/147): libgfortran-8.5.0-20.el8.x86_64.rpm 11 MB/s | 645 kB 00:00 2024-01-16T14:51:57.999 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2024-01-16T14:51:57.999 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2024-01-16T14:51:58.020 INFO:teuthology.orchestra.run.smithi026.stdout:(89/147): libquadmath-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 172 kB 00:00 2024-01-16T14:51:58.046 INFO:teuthology.orchestra.run.smithi026.stdout:(90/147): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.9 MB/s | 100 kB 00:00 2024-01-16T14:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout:(91/147): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2024-01-16T14:51:58.133 INFO:teuthology.orchestra.run.smithi026.stdout:(92/147): python3-cryptography-3.2.1-6.el8.x86_ 6.2 MB/s | 559 kB 00:00 2024-01-16T14:51:58.155 INFO:teuthology.orchestra.run.smithi026.stdout:(93/147): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 868 kB/s | 73 kB 00:00 2024-01-16T14:51:58.180 INFO:teuthology.orchestra.run.smithi026.stdout:(94/147): elfutils-libelf-devel-0.189-3.el8.x86 1.3 MB/s | 62 kB 00:00 2024-01-16T14:51:58.205 INFO:teuthology.orchestra.run.smithi026.stdout:(95/147): librabbitmq-0.9.0-4.el8.x86_64.rpm 934 kB/s | 47 kB 00:00 2024-01-16T14:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout:(96/147): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 8.2 MB/s | 562 kB 00:00 2024-01-16T14:51:58.281 INFO:teuthology.orchestra.run.smithi026.stdout:(97/147): NetworkManager-initscripts-updown-1.4 4.3 MB/s | 144 kB 00:00 2024-01-16T14:51:58.315 INFO:teuthology.orchestra.run.smithi026.stdout:(98/147): lua-devel-5.3.4-12.el8.x86_64.rpm 905 kB/s | 30 kB 00:00 2024-01-16T14:51:58.365 INFO:teuthology.orchestra.run.smithi026.stdout:(99/147): openssl-devel-1.1.1k-12.el8_9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-01-16T14:51:58.390 INFO:teuthology.orchestra.run.smithi026.stdout:(100/147): python3-jsonpointer-1.10-11.el8.noar 265 kB/s | 20 kB 00:00 2024-01-16T14:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:(101/147): python3-prettytable-0.7.2-14.el8.noa 885 kB/s | 44 kB 00:00 2024-01-16T14:51:58.441 INFO:teuthology.orchestra.run.smithi026.stdout:(102/147): python3-pyasn1-modules-0.3.7-6.el8.n 2.1 MB/s | 110 kB 00:00 2024-01-16T14:51:58.457 INFO:teuthology.orchestra.run.smithi026.stdout:(103/147): python3-jsonpatch-1.21-2.el8.noarch. 634 kB/s | 27 kB 00:00 2024-01-16T14:51:58.508 INFO:teuthology.orchestra.run.smithi026.stdout:(104/147): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.8 MB/s | 126 kB 00:00 2024-01-16T14:51:58.542 INFO:teuthology.orchestra.run.smithi026.stdout:(105/147): python3-werkzeug-0.12.2-4.el8.noarch 5.3 MB/s | 457 kB 00:00 2024-01-16T14:51:58.575 INFO:teuthology.orchestra.run.smithi026.stdout:(106/147): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2024-01-16T14:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout:(107/147): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2024-01-16T14:51:58.643 INFO:teuthology.orchestra.run.smithi026.stdout:(108/147): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2024-01-16T14:51:58.659 INFO:teuthology.orchestra.run.smithi026.stdout:(109/147): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2024-01-16T14:51:58.684 INFO:teuthology.orchestra.run.smithi026.stdout:(110/147): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2024-01-16T14:51:58.710 INFO:teuthology.orchestra.run.smithi026.stdout:(111/147): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-01-16T14:51:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:(112/147): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-01-16T14:51:58.760 INFO:teuthology.orchestra.run.smithi026.stdout:(113/147): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2024-01-16T14:51:58.811 INFO:teuthology.orchestra.run.smithi026.stdout:(114/147): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2024-01-16T14:51:58.928 INFO:teuthology.orchestra.run.smithi026.stdout:(115/147): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2024-01-16T14:51:59.003 INFO:teuthology.orchestra.run.smithi026.stdout:(116/147): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2024-01-16T14:51:59.029 INFO:teuthology.orchestra.run.smithi026.stdout:(117/147): socat-1.7.4.1-1.el8.x86_64.rpm 3.1 MB/s | 323 kB 00:00 2024-01-16T14:51:59.054 INFO:teuthology.orchestra.run.smithi026.stdout:(118/147): python3-numpy-f2py-1.14.3-10.el8.x86 4.4 MB/s | 225 kB 00:00 2024-01-16T14:51:59.088 INFO:teuthology.orchestra.run.smithi026.stdout:(119/147): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2024-01-16T14:51:59.208 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2024-01-16T14:51:59.218 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2024-01-16T14:51:59.396 INFO:teuthology.orchestra.run.smithi026.stdout:(120/147): python3-mako-1.0.6-14.el8.noarch.rpm 509 kB/s | 157 kB 00:00 2024-01-16T14:51:59.530 INFO:teuthology.orchestra.run.smithi026.stdout:(121/147): openblas-threads-0.3.15-6.el8.x86_64 11 MB/s | 5.2 MB 00:00 2024-01-16T14:51:59.580 INFO:teuthology.orchestra.run.smithi026.stdout:(122/147): automake-1.16.1-8.el8.noarch.rpm 14 MB/s | 714 kB 00:00 2024-01-16T14:51:59.670 INFO:teuthology.orchestra.run.smithi026.stdout:(123/147): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-01-16T14:51:59.697 INFO:teuthology.orchestra.run.smithi026.stdout:(124/147): libpq-13.11-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2024-01-16T14:51:59.722 INFO:teuthology.orchestra.run.smithi026.stdout:(125/147): jq-1.6-7.el8.x86_64.rpm 3.8 MB/s | 202 kB 00:00 2024-01-16T14:51:59.748 INFO:teuthology.orchestra.run.smithi026.stdout:(126/147): python3-pytz-2017.2-11.el8.noarch.rp 1.1 MB/s | 54 kB 00:00 2024-01-16T14:51:59.798 INFO:teuthology.orchestra.run.smithi026.stdout:(127/147): postgresql-10.23-3.module+el8.9.0+20 20 MB/s | 1.5 MB 00:00 2024-01-16T14:51:59.832 INFO:teuthology.orchestra.run.smithi026.stdout:(128/147): postgresql-contrib-10.23-3.module+el 9.5 MB/s | 811 kB 00:00 2024-01-16T14:51:59.982 INFO:teuthology.orchestra.run.smithi026.stdout:(129/147): postgresql-server-10.23-3.module+el8 28 MB/s | 5.1 MB 00:00 2024-01-16T14:52:00.233 INFO:teuthology.orchestra.run.smithi026.stdout:(130/147): python3-scipy-1.0.0-21.module+el8.9. 34 MB/s | 14 MB 00:00 2024-01-16T14:52:00.433 INFO:teuthology.orchestra.run.smithi026.stdout:(131/147): librados2-18.0.0-7629.g73a4532f.el8. 8.4 MB/s | 3.8 MB 00:00 2024-01-16T14:52:00.474 INFO:teuthology.orchestra.run.smithi026.stdout:(132/147): elfutils-libelf-0.189-3.el8.x86_64.r 5.5 MB/s | 232 kB 00:00 2024-01-16T14:52:00.508 INFO:teuthology.orchestra.run.smithi026.stdout:(133/147): elfutils-libs-0.189-3.el8.x86_64.rpm 8.9 MB/s | 303 kB 00:00 2024-01-16T14:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:(134/147): libmount-2.32.1-43.el8.x86_64.rpm 5.6 MB/s | 237 kB 00:00 2024-01-16T14:52:00.592 INFO:teuthology.orchestra.run.smithi026.stdout:(135/147): libsmartcols-2.32.1-43.el8.x86_64.rp 4.2 MB/s | 179 kB 00:00 2024-01-16T14:52:00.626 INFO:teuthology.orchestra.run.smithi026.stdout:(136/147): libuuid-2.32.1-43.el8.x86_64.rpm 2.9 MB/s | 99 kB 00:00 2024-01-16T14:52:00.726 INFO:teuthology.orchestra.run.smithi026.stdout:(137/147): librbd1-18.0.0-7629.g73a4532f.el8.x8 7.5 MB/s | 3.7 MB 00:00 2024-01-16T14:52:00.801 INFO:teuthology.orchestra.run.smithi026.stdout:(138/147): util-linux-2.32.1-43.el8.x86_64.rpm 14 MB/s | 2.5 MB 00:00 2024-01-16T14:52:00.827 INFO:teuthology.orchestra.run.smithi026.stdout:(139/147): util-linux-user-2.32.1-43.el8.x86_64 1.0 MB/s | 103 kB 00:00 2024-01-16T14:52:00.877 INFO:teuthology.orchestra.run.smithi026.stdout:(140/147): iproute-6.2.0-5.el8_9.x86_64.rpm 11 MB/s | 854 kB 00:00 2024-01-16T14:52:00.902 INFO:teuthology.orchestra.run.smithi026.stdout:(141/147): libblkid-2.32.1-43.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-01-16T14:52:00.927 INFO:teuthology.orchestra.run.smithi026.stdout:(142/147): libfdisk-2.32.1-43.el8.x86_64.rpm 4.9 MB/s | 254 kB 00:00 2024-01-16T14:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout:(143/147): xfsprogs-5.0.0-12.el8.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-01-16T14:52:01.003 INFO:teuthology.orchestra.run.smithi026.stdout:(144/147): openssl-1.1.1k-12.el8_9.x86_64.rpm 9.2 MB/s | 711 kB 00:00 2024-01-16T14:52:01.054 INFO:teuthology.orchestra.run.smithi026.stdout:(145/147): xfsdump-3.1.8-7.el8_9.x86_64.rpm 6.5 MB/s | 333 kB 00:00 2024-01-16T14:52:01.095 INFO:teuthology.orchestra.run.smithi026.stdout:(146/147): openssl-libs-1.1.1k-12.el8_9.x86_64. 12 MB/s | 1.5 MB 00:00 2024-01-16T14:52:01.379 INFO:teuthology.orchestra.run.smithi026.stdout:(147/147): ceph-osd-18.0.0-7629.g73a4532f.el8.x 1.6 MB/s | 18 MB 00:11 2024-01-16T14:52:01.382 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-01-16T14:52:01.383 INFO:teuthology.orchestra.run.smithi026.stdout:Total 16 MB/s | 218 MB 00:13 2024-01-16T14:52:02.485 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-01-16T14:52:02.935 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-01-16T14:52:02.936 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-01-16T14:52:04.273 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-01-16T14:52:04.285 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-01-16T14:52:05.120 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2024-01-16T14:52:05.255 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-16T14:52:05.532 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-16T14:52:05.728 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-16T14:52:05.892 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/163 2024-01-16T14:52:06.079 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-16T14:52:06.147 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-16T14:52:06.371 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-16T14:52:06.442 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-16T14:52:06.675 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/163 2024-01-16T14:52:06.902 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/163 2024-01-16T14:52:07.096 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-16T14:52:07.172 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-16T14:52:07.387 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-16T14:52:07.793 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-16T14:52:08.020 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:52:08.411 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:52:08.595 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:52:08.754 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:52:09.219 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:52:09.605 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:52:10.276 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-01-16T14:52:10.402 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 12/163 2024-01-16T14:52:10.427 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-01-16T14:52:10.578 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:52:10.668 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-16T14:52:10.764 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:52:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/163 2024-01-16T14:52:11.003 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-16T14:52:11.085 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-16T14:52:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/163 2024-01-16T14:52:11.261 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-16T14:52:11.284 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpq-13.11-1.el8.x86_64 15/163 2024-01-16T14:52:11.341 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/163 2024-01-16T14:52:11.544 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-16T14:52:11.557 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/163 2024-01-16T14:52:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/163 2024-01-16T14:52:11.764 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/163 2024-01-16T14:52:11.766 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/163 2024-01-16T14:52:11.988 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/163 2024-01-16T14:52:12.136 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-16T14:52:12.149 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/163 2024-01-16T14:52:12.262 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/163 2024-01-16T14:52:12.376 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/163 2024-01-16T14:52:12.436 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-16T14:52:12.561 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 20/163 2024-01-16T14:52:12.672 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 20/163 2024-01-16T14:52:12.836 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 21/163 2024-01-16T14:52:12.869 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/163 2024-01-16T14:52:12.941 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 21/163 2024-01-16T14:52:13.086 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:52:13.123 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 22/163 2024-01-16T14:52:13.413 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/163 2024-01-16T14:52:13.536 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:52:13.658 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/163 2024-01-16T14:52:13.736 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:52:13.869 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-16T14:52:13.878 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:52:14.004 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-16T14:52:14.060 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:52:14.149 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:52:14.202 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-16T14:52:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-16T14:52:14.981 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 12/163 2024-01-16T14:52:15.086 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-16T14:52:15.143 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:52:15.328 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:52:15.674 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-16T14:52:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-16T14:52:15.894 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.15-6.el8.x86_64 28/163 2024-01-16T14:52:15.940 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/163 2024-01-16T14:52:16.120 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpq-13.11-1.el8.x86_64 15/163 2024-01-16T14:52:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 16/163 2024-01-16T14:52:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 17/163 2024-01-16T14:52:16.578 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 28/163 2024-01-16T14:52:16.883 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 29/163 2024-01-16T14:52:16.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 18/163 2024-01-16T14:52:17.073 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 30/163 2024-01-16T14:52:17.160 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 19/163 2024-01-16T14:52:17.327 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 20/163 2024-01-16T14:52:17.345 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/163 2024-01-16T14:52:17.428 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 20/163 2024-01-16T14:52:17.550 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g73a4532f.el8. 32/163 2024-01-16T14:52:17.560 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 21/163 2024-01-16T14:52:17.645 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 21/163 2024-01-16T14:52:17.730 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 33/163 2024-01-16T14:52:17.849 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 22/163 2024-01-16T14:52:18.060 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_6 34/163 2024-01-16T14:52:18.150 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/163 2024-01-16T14:52:18.294 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-16T14:52:18.367 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/163 2024-01-16T14:52:18.560 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-16T14:52:18.702 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 25/163 2024-01-16T14:52:18.859 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-16T14:52:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 26/163 2024-01-16T14:52:19.709 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-16T14:52:20.029 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-16T14:52:20.380 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 27/163 2024-01-16T14:52:20.429 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 36/163 2024-01-16T14:52:20.559 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.15-6.el8.x86_64 28/163 2024-01-16T14:52:21.331 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 28/163 2024-01-16T14:52:21.404 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 37/163 2024-01-16T14:52:21.575 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 29/163 2024-01-16T14:52:21.773 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 38/163 2024-01-16T14:52:21.808 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 30/163 2024-01-16T14:52:22.027 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 39/163 2024-01-16T14:52:22.045 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 31/163 2024-01-16T14:52:22.225 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g73a4532f.el8. 32/163 2024-01-16T14:52:22.288 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/163 2024-01-16T14:52:22.381 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 33/163 2024-01-16T14:52:22.496 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/163 2024-01-16T14:52:22.714 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_6 34/163 2024-01-16T14:52:22.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 42/163 2024-01-16T14:52:22.925 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-16T14:52:22.999 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 43/163 2024-01-16T14:52:23.230 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_6 44/163 2024-01-16T14:52:23.476 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/163 2024-01-16T14:52:23.558 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/163 2024-01-16T14:52:23.784 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-16T14:52:23.858 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-16T14:52:24.098 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/163 2024-01-16T14:52:24.337 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/163 2024-01-16T14:52:24.576 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-16T14:52:24.884 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/163 2024-01-16T14:52:25.017 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-16T14:52:25.256 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 36/163 2024-01-16T14:52:25.426 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-16T14:52:25.520 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-16T14:52:25.760 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.1-8.el8.noarch 51/163 2024-01-16T14:52:25.992 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/163 2024-01-16T14:52:26.074 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/163 2024-01-16T14:52:26.284 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-16T14:52:26.335 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 37/163 2024-01-16T14:52:26.367 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-16T14:52:26.592 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-7.el8.x86_64 54/163 2024-01-16T14:52:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 38/163 2024-01-16T14:52:26.774 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 55/163 2024-01-16T14:52:26.941 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 39/163 2024-01-16T14:52:26.992 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 56/163 2024-01-16T14:52:27.152 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 57/163 2024-01-16T14:52:27.186 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el8.noarch 40/163 2024-01-16T14:52:27.363 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 41/163 2024-01-16T14:52:27.413 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.7-17.el8.x86_64 58/163 2024-01-16T14:52:27.623 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 42/163 2024-01-16T14:52:27.624 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 59/163 2024-01-16T14:52:27.822 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 60/163 2024-01-16T14:52:27.824 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 43/163 2024-01-16T14:52:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_6 44/163 2024-01-16T14:52:28.043 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 61/163 2024-01-16T14:52:28.258 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/163 2024-01-16T14:52:28.271 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/163 2024-01-16T14:52:28.330 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/163 2024-01-16T14:52:28.541 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-16T14:52:28.610 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 63/163 2024-01-16T14:52:28.630 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 46/163 2024-01-16T14:52:28.834 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 47/163 2024-01-16T14:52:28.907 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/163 2024-01-16T14:52:29.109 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 48/163 2024-01-16T14:52:29.130 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/163 2024-01-16T14:52:29.279 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 66/163 2024-01-16T14:52:29.357 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-16T14:52:29.519 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 67/163 2024-01-16T14:52:29.845 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 49/163 2024-01-16T14:52:29.973 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 68/163 2024-01-16T14:52:30.300 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-16T14:52:30.335 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 69/163 2024-01-16T14:52:30.409 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 50/163 2024-01-16T14:52:30.561 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 70/163 2024-01-16T14:52:30.668 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.1-8.el8.noarch 51/163 2024-01-16T14:52:30.812 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 71/163 2024-01-16T14:52:30.866 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/163 2024-01-16T14:52:30.957 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/163 2024-01-16T14:52:31.087 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 72/163 2024-01-16T14:52:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-16T14:52:31.206 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 53/163 2024-01-16T14:52:31.392 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-7.el8.x86_64 54/163 2024-01-16T14:52:31.400 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 73/163 2024-01-16T14:52:31.516 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 55/163 2024-01-16T14:52:31.641 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 74/163 2024-01-16T14:52:31.725 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 56/163 2024-01-16T14:52:31.825 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 75/163 2024-01-16T14:52:31.887 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 57/163 2024-01-16T14:52:32.060 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 76/163 2024-01-16T14:52:32.068 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.3.4-12.el8.x86_64 58/163 2024-01-16T14:52:32.254 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.7-17.el8.x86_64 59/163 2024-01-16T14:52:32.254 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 77/163 2024-01-16T14:52:32.500 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 60/163 2024-01-16T14:52:32.562 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 78/163 2024-01-16T14:52:32.692 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lua-devel-5.3.4-12.el8.x86_64 79/163 2024-01-16T14:52:32.714 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/163 2024-01-16T14:52:32.891 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 80/163 2024-01-16T14:52:32.913 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/163 2024-01-16T14:52:33.112 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 81/163 2024-01-16T14:52:33.155 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/163 2024-01-16T14:52:33.419 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 82/163 2024-01-16T14:52:33.475 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 64/163 2024-01-16T14:52:33.656 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 83/163 2024-01-16T14:52:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 84/163 2024-01-16T14:52:33.913 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 65/163 2024-01-16T14:52:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 85/163 2024-01-16T14:52:34.113 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 66/163 2024-01-16T14:52:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 86/163 2024-01-16T14:52:34.301 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 67/163 2024-01-16T14:52:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 87/163 2024-01-16T14:52:34.518 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 68/163 2024-01-16T14:52:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 88/163 2024-01-16T14:52:34.928 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 69/163 2024-01-16T14:52:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 89/163 2024-01-16T14:52:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 90/163 2024-01-16T14:52:35.380 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 70/163 2024-01-16T14:52:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 91/163 2024-01-16T14:52:35.556 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 71/163 2024-01-16T14:52:35.677 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 92/163 2024-01-16T14:52:35.824 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 72/163 2024-01-16T14:52:35.888 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 93/163 2024-01-16T14:52:36.092 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/163 2024-01-16T14:52:36.175 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-2.6-1.el8.noarch 94/163 2024-01-16T14:52:36.400 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/163 2024-01-16T14:52:36.414 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 95/163 2024-01-16T14:52:36.631 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 96/163 2024-01-16T14:52:36.641 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 75/163 2024-01-16T14:52:36.835 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 76/163 2024-01-16T14:52:36.898 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 97/163 2024-01-16T14:52:37.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 77/163 2024-01-16T14:52:37.255 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 78/163 2024-01-16T14:52:37.543 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 79/163 2024-01-16T14:52:37.683 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 80/163 2024-01-16T14:52:37.898 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 81/163 2024-01-16T14:52:37.964 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 98/163 2024-01-16T14:52:38.153 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 82/163 2024-01-16T14:52:38.277 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 99/163 2024-01-16T14:52:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 83/163 2024-01-16T14:52:38.514 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 100/163 2024-01-16T14:52:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 84/163 2024-01-16T14:52:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 85/163 2024-01-16T14:52:38.873 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 101/163 2024-01-16T14:52:39.053 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 86/163 2024-01-16T14:52:39.218 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 102/163 2024-01-16T14:52:39.271 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 87/163 2024-01-16T14:52:39.499 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : luarocks-3.5.0-1.el8.noarch 103/163 2024-01-16T14:52:39.544 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 88/163 2024-01-16T14:52:39.698 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 104/163 2024-01-16T14:52:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 89/163 2024-01-16T14:52:39.888 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 90/163 2024-01-16T14:52:39.924 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 105/163 2024-01-16T14:52:40.141 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 91/163 2024-01-16T14:52:40.336 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 92/163 2024-01-16T14:52:40.521 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 93/163 2024-01-16T14:52:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-2.6-1.el8.noarch 94/163 2024-01-16T14:52:40.973 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 95/163 2024-01-16T14:52:41.182 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 96/163 2024-01-16T14:52:41.410 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 97/163 2024-01-16T14:52:42.726 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 98/163 2024-01-16T14:52:43.084 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 99/163 2024-01-16T14:52:43.294 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 100/163 2024-01-16T14:52:43.654 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 101/163 2024-01-16T14:52:43.979 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 102/163 2024-01-16T14:52:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.5.0-1.el8.noarch 103/163 2024-01-16T14:52:44.523 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:44.593 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 104/163 2024-01-16T14:52:44.781 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 105/163 2024-01-16T14:52:44.790 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:45.365 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:45.676 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 107/163 2024-01-16T14:52:45.762 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 107/163 2024-01-16T14:52:45.967 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 108/163 2024-01-16T14:52:49.041 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:49.247 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:49.890 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 106/163 2024-01-16T14:52:50.170 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 107/163 2024-01-16T14:52:50.249 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 107/163 2024-01-16T14:52:50.452 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 108/163 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 108/163 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /run 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2024-01-16T14:53:05.255 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2024-01-16T14:53:05.256 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2024-01-16T14:53:05.256 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2024-01-16T14:53:05.256 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:05.637 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 109/163 2024-01-16T14:53:07.228 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 109/163 2024-01-16T14:53:07.456 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 110/163 2024-01-16T14:53:07.677 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 110/163 2024-01-16T14:53:07.861 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 111/163 2024-01-16T14:53:08.319 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 111/163 2024-01-16T14:53:08.503 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 112/163 2024-01-16T14:53:08.520 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 112/163 2024-01-16T14:53:10.330 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 113/163 2024-01-16T14:53:10.525 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 113/163 2024-01-16T14:53:10.643 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 114/163 2024-01-16T14:53:10.852 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 115/163 2024-01-16T14:53:11.110 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 116/163 2024-01-16T14:53:11.330 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 116/163 2024-01-16T14:53:11.597 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 117/163 2024-01-16T14:53:12.562 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 117/163 2024-01-16T14:53:12.652 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 108/163 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /run 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2024-01-16T14:53:12.653 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:12.769 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 118/163 2024-01-16T14:53:12.863 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 118/163 2024-01-16T14:53:13.018 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 109/163 2024-01-16T14:53:13.255 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 119/163 2024-01-16T14:53:13.480 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 120/163 2024-01-16T14:53:13.518 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 120/163 2024-01-16T14:53:13.702 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 121/163 2024-01-16T14:53:13.714 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 121/163 2024-01-16T14:53:14.789 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 109/163 2024-01-16T14:53:14.958 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 110/163 2024-01-16T14:53:15.049 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 122/163 2024-01-16T14:53:15.205 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 110/163 2024-01-16T14:53:15.254 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 123/163 2024-01-16T14:53:15.289 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 123/163 2024-01-16T14:53:15.350 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 111/163 2024-01-16T14:53:15.435 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 124/163 2024-01-16T14:53:15.478 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g73a45 124/163 2024-01-16T14:53:15.652 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 125/163 2024-01-16T14:53:15.861 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 111/163 2024-01-16T14:53:16.008 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 125/163 2024-01-16T14:53:16.026 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 112/163 2024-01-16T14:53:16.048 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 112/163 2024-01-16T14:53:16.185 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 126/163 2024-01-16T14:53:18.061 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 113/163 2024-01-16T14:53:18.235 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 113/163 2024-01-16T14:53:18.351 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 114/163 2024-01-16T14:53:18.561 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 115/163 2024-01-16T14:53:18.791 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 116/163 2024-01-16T14:53:19.034 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 116/163 2024-01-16T14:53:19.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 117/163 2024-01-16T14:53:20.371 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 117/163 2024-01-16T14:53:20.516 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 118/163 2024-01-16T14:53:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 118/163 2024-01-16T14:53:20.845 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 126/163 2024-01-16T14:53:20.991 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 119/163 2024-01-16T14:53:21.182 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 120/163 2024-01-16T14:53:21.227 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 120/163 2024-01-16T14:53:21.399 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 121/163 2024-01-16T14:53:21.415 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 121/163 2024-01-16T14:53:21.873 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 127/163 2024-01-16T14:53:22.172 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/163 2024-01-16T14:53:22.496 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/163 2024-01-16T14:53:22.886 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 130/163 2024-01-16T14:53:22.893 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 122/163 2024-01-16T14:53:23.048 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 123/163 2024-01-16T14:53:23.089 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 123/163 2024-01-16T14:53:23.218 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 131/163 2024-01-16T14:53:23.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 124/163 2024-01-16T14:53:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g73a45 124/163 2024-01-16T14:53:23.442 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-25.el8.x86_64 132/163 2024-01-16T14:53:23.457 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 132/163 2024-01-16T14:53:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 125/163 2024-01-16T14:53:23.694 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 133/163 2024-01-16T14:53:23.885 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 125/163 2024-01-16T14:53:23.896 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 134/163 2024-01-16T14:53:24.049 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 126/163 2024-01-16T14:53:24.076 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_ 135/163 2024-01-16T14:53:26.355 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:26.667 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:26.705 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:27.196 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 137/163 2024-01-16T14:53:27.615 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 138/163 2024-01-16T14:53:27.854 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 139/163 2024-01-16T14:53:28.033 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 140/163 2024-01-16T14:53:28.283 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 141/163 2024-01-16T14:53:28.602 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 142/163 2024-01-16T14:53:28.952 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 143/163 2024-01-16T14:53:29.165 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/163 2024-01-16T14:53:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 126/163 2024-01-16T14:53:29.242 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/163 2024-01-16T14:53:29.474 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/163 2024-01-16T14:53:29.487 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/163 2024-01-16T14:53:29.649 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 146/163 2024-01-16T14:53:29.658 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 146/163 2024-01-16T14:53:29.826 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/163 2024-01-16T14:53:30.031 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 148/163 2024-01-16T14:53:30.253 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 149/163 2024-01-16T14:53:30.299 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 127/163 2024-01-16T14:53:30.386 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 150/163 2024-01-16T14:53:30.386 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-16T14:53:30.612 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/163 2024-01-16T14:53:30.686 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-16T14:53:30.686 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-16T14:53:30.911 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-16T14:53:30.911 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-16T14:53:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/163 2024-01-16T14:53:31.197 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-16T14:53:31.364 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 130/163 2024-01-16T14:53:31.443 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 154/163 2024-01-16T14:53:31.658 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 155/163 2024-01-16T14:53:31.708 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 131/163 2024-01-16T14:53:31.901 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 156/163 2024-01-16T14:53:31.901 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-16T14:53:31.914 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-25.el8.x86_64 132/163 2024-01-16T14:53:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 132/163 2024-01-16T14:53:32.128 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 133/163 2024-01-16T14:53:32.160 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-16T14:53:32.160 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 158/163 2024-01-16T14:53:32.344 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 134/163 2024-01-16T14:53:32.435 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 158/163 2024-01-16T14:53:32.435 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-16T14:53:32.498 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_ 135/163 2024-01-16T14:53:32.701 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-16T14:53:32.702 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-16T14:53:32.968 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-16T14:53:32.968 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-16T14:53:33.210 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 162/163 2024-01-16T14:53:33.376 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-16T14:53:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 1/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 2/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 3/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 4/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 5/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 6/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 7/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 8/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:53:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 12/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_ 14/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 15/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 16/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_6 17/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 18/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 19/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 20/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g73a4532f.el8. 21/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_6 22/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 23/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 24/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 25/163 2024-01-16T14:53:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 26/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 27/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 28/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 29/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 30/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 31/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 32/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 33/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 34/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 35/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 36/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 37/163 2024-01-16T14:53:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 38/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/163 2024-01-16T14:53:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-16T14:53:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lua-devel-5.3.4-12.el8.x86_64 73/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 74/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 76/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 77/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 79/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 80/163 2024-01-16T14:53:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 82/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 83/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 84/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 85/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 86/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 87/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 88/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 89/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 90/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 91/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 92/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 93/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 94/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 95/163 2024-01-16T14:53:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 99/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 100/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 101/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 102/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 103/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 104/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 106/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.0.4-10.el8.x86_64 107/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.1-9.el8.x86_64 108/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 109/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 110/163 2024-01-16T14:53:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 111/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 112/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 113/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 114/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 115/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 116/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 117/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 118/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 119/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 120/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 121/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 122/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 123/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.1-8.el8.noarch 124/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpq-13.11-1.el8.x86_64 125/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-7.el8.x86_64 126/163 2024-01-16T14:53:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 127/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 128/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 130/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 131/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 132/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 133/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 134/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 135/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 136/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 137/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 138/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 139/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 140/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 141/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 142/163 2024-01-16T14:53:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 143/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 144/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 145/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 146/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 147/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 148/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 149/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 150/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 151/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 152/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 153/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 154/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 155/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 156/163 2024-01-16T14:53:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 157/163 2024-01-16T14:53:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 158/163 2024-01-16T14:53:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 159/163 2024-01-16T14:53:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 160/163 2024-01-16T14:53:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/163 2024-01-16T14:53:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 162/163 2024-01-16T14:53:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:35.347 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:35.390 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 136/163 2024-01-16T14:53:35.819 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 137/163 2024-01-16T14:53:36.176 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 138/163 2024-01-16T14:53:36.385 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 139/163 2024-01-16T14:53:36.556 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 140/163 2024-01-16T14:53:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 141/163 2024-01-16T14:53:37.050 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 142/163 2024-01-16T14:53:37.313 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 143/163 2024-01-16T14:53:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/163 2024-01-16T14:53:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/163 2024-01-16T14:53:37.821 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/163 2024-01-16T14:53:37.837 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/163 2024-01-16T14:53:37.996 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 146/163 2024-01-16T14:53:38.008 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 146/163 2024-01-16T14:53:38.183 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 147/163 2024-01-16T14:53:38.342 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 148/163 2024-01-16T14:53:38.557 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 149/163 2024-01-16T14:53:38.693 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 150/163 2024-01-16T14:53:38.693 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-16T14:53:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 151/163 2024-01-16T14:53:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-16T14:53:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 152/163 2024-01-16T14:53:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-16T14:53:39.454 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 153/163 2024-01-16T14:53:39.706 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 154/163 2024-01-16T14:53:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 155/163 2024-01-16T14:53:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 156/163 2024-01-16T14:53:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-16T14:53:40.291 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 157/163 2024-01-16T14:53:40.291 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 158/163 2024-01-16T14:53:40.532 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 158/163 2024-01-16T14:53:40.532 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-16T14:53:40.655 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 163/163 2024-01-16T14:53:40.655 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2024-01-16T14:53:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 159/163 2024-01-16T14:53:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-16T14:53:40.982 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 160/163 2024-01-16T14:53:40.982 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-16T14:53:41.240 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 161/163 2024-01-16T14:53:41.399 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 162/163 2024-01-16T14:53:41.399 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.992 INFO:teuthology.orchestra.run.smithi093.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-16T14:53:41.993 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.1-8.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.0.4-10.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.994 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.1-9.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-16T14:53:41.995 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-7.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libpq-13.11-1.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-16T14:53:41.996 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: lua-devel-5.3.4-12.el8.x86_64 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: luarocks-3.5.0-1.el8.noarch 2024-01-16T14:53:41.997 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-16T14:53:41.998 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-16T14:53:41.999 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-2.6-1.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-16T14:53:42.000 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-16T14:53:42.001 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-43.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-16T14:53:42.002 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-16T14:53:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-16T14:53:42.003 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:42.003 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2024-01-16T14:53:42.204 DEBUG:teuthology.parallel:result is None 2024-01-16T14:53:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 163/163 2024-01-16T14:53:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 1/163 2024-01-16T14:53:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 2/163 2024-01-16T14:53:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 3/163 2024-01-16T14:53:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 4/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g73a45 5/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 6/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 7/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 8/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 9/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 10/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 11/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 12/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 13/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_ 14/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 15/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 16/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_6 17/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86 18/163 2024-01-16T14:53:42.377 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 19/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el 20/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g73a4532f.el8. 21/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_6 22/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 23/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 24/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 25/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 26/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 27/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 28/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f. 29/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noa 30/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.n 31/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a 32/163 2024-01-16T14:53:42.378 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el 33/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 34/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.e 35/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 36/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 37/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 38/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/163 2024-01-16T14:53:42.379 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/163 2024-01-16T14:53:42.380 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 73/163 2024-01-16T14:53:42.381 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 78/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 80/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 82/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 83/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 84/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 85/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 86/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 87/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/163 2024-01-16T14:53:42.382 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 96/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 97/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 98/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 99/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.3.4-12.el8.x86_64 100/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 101/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 102/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 103/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 104/163 2024-01-16T14:53:42.383 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 106/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.0.4-10.el8.x86_64 107/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.1-9.el8.x86_64 108/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 109/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 110/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 111/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 112/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 113/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 114/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 115/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 116/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 117/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 118/163 2024-01-16T14:53:42.384 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 119/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 120/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 121/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 122/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 123/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.1-8.el8.noarch 124/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpq-13.11-1.el8.x86_64 125/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-7.el8.x86_64 126/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 127/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 128/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 130/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 131/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 132/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 133/163 2024-01-16T14:53:42.385 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 134/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 135/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 136/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 137/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 138/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 139/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 140/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 141/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 142/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 143/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 144/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 145/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 146/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 147/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 148/163 2024-01-16T14:53:42.386 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 149/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 150/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 151/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 152/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 153/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 154/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 155/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 156/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 157/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 158/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 159/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 160/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/163 2024-01-16T14:53:42.387 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 162/163 2024-01-16T14:53:49.359 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 163/163 2024-01-16T14:53:49.359 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2024-01-16T14:53:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:50.024 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-01-16T14:53:50.024 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: libblkid-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: libmount-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.025 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.1-8.el8.noarch 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.0.4-10.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.026 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.027 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:18.0.0-7629.g73a4532f.el8.noarch 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.1-9.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.7-17.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-7.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.028 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.2-3.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libpq-13.11-1.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-01-16T14:53:50.029 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-25.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.3.1-3.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.3.4-12.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.5.0-1.el8.noarch 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.15-6.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-01-16T14:53:50.030 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.5.1-7.el8.noarch 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-01-16T14:53:50.031 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-6.2-6.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-01-16T14:53:50.032 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.0.6-14.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-2.6-1.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.14-14.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2017.2-11.el8.noarch 2024-01-16T14:53:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.4.1-12.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-01-16T14:53:50.034 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.0.0-7629.g73a4532f.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-1.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.13.0-2.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-43.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-01-16T14:53:50.035 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-01-16T14:53:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-01-16T14:53:50.036 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:50.036 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-01-16T14:53:50.380 DEBUG:teuthology.parallel:result is None 2024-01-16T14:53:50.381 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:53:50.614 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-16T14:53:50.644 INFO:teuthology.orchestra.run.smithi026.stdout:18.0.0-7629.g73a4532f.el8 2024-01-16T14:53:50.644 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g73a4532f.el8 2024-01-16T14:53:50.645 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g73a4532f is installed. 2024-01-16T14:53:50.646 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=73a4532fc4d97adee0b985d4d42874ea351a1c86 2024-01-16T14:53:50.865 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-16T14:53:50.894 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-7629.g73a4532f.el8 2024-01-16T14:53:50.894 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g73a4532f.el8 2024-01-16T14:53:50.894 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g73a4532f is installed. 2024-01-16T14:53:50.896 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-16T14:53:50.896 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:50.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-16T14:53:50.948 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:50.949 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-16T14:53:50.998 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-16T14:53:50.999 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:50.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-01-16T14:53:51.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-16T14:53:51.110 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:51.110 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2024-01-16T14:53:51.143 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-16T14:53:51.216 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-16T14:53:51.216 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:51.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-16T14:53:51.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-16T14:53:51.325 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:51.325 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-16T14:53:51.357 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-16T14:53:51.427 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-16T14:53:51.427 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:51.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-01-16T14:53:51.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-16T14:53:51.538 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:51.538 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2024-01-16T14:53:51.569 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-16T14:53:51.639 INFO:teuthology.run_tasks:Running task ceph... 2024-01-16T14:53:51.724 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-16T14:53:51.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-01-16T14:53:51.727 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /var/log/ceph 2024-01-16T14:53:51.759 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-16T14:53:51.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-16T14:53:51.802 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-16T14:53:51.834 INFO:tasks.ceph:Creating extra log directories... 2024-01-16T14:53:51.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-16T14:53:51.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-16T14:53:51.915 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-16T14:53:51.915 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-16T14:53:51.915 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-16T14:53:51.915 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:51.915 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-16T14:53:51.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-16T14:53:52.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-16T14:53:52.133 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:52.133 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-16T14:53:52.152 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-16T14:53:52.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-16T14:53:52.319 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-16T14:53:52.319 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-16T14:53:52.319 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi/7518190', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7518190', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-01-16_14:37:50-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-clone-stats', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 2668, 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-clone-stats', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_73a4532fc4d97adee0b985d4d42874ea351a1c86/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJM9PxVisPRU+UgAr+oU+jzlLJ7dCgKZNi1khVAIbHVx8jd9XEJwPTikMbXnSCUlaYnLUNEiPuG8mb21kVYz1U=', 'smithi093.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBoC4AG+9lTpJtSCEYm+u6Kx2a7PIsQRMQmaTguTE0kBsS0dDi4YYwYzCH2A/BtKdyJG1YXrZcYHBX/glEVWOA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '73a4532fc4d97adee0b985d4d42874ea351a1c86', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgress']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-16_14:37:50', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-16T14:53:52.319 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-16T14:53:52.322 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-16T14:53:52.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-01-16T14:53:52.384 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m0777 -- /var/run/ceph 2024-01-16T14:53:52.439 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:52.439 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-01-16T14:53:52.479 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-16T14:53:52.479 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-01-16T14:53:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-16T14:53:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:52.541 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 172019 Links: 1 2024-01-16T14:53:52.541 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:52.541 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:52.542 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-16 14:53:41.960414986 +0000 2024-01-16T14:53:52.542 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-16 14:50:36.345796186 +0000 2024-01-16T14:53:52.542 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-16 14:50:36.345796186 +0000 2024-01-16T14:53:52.542 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-01-16T14:53:52.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-16T14:53:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-16T14:53:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-16T14:53:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.00013025 s, 3.9 MB/s 2024-01-16T14:53:52.616 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-16T14:53:52.680 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 177467 Links: 1 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-16 14:53:41.960414986 +0000 2024-01-16T14:53:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-16 14:50:36.704790285 +0000 2024-01-16T14:53:52.743 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-16 14:50:36.704790285 +0000 2024-01-16T14:53:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-01-16T14:53:52.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-16T14:53:52.816 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-16T14:53:52.816 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-16T14:53:52.816 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000130907 s, 3.9 MB/s 2024-01-16T14:53:52.817 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-16T14:53:52.880 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-01-16T14:53:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-16T14:53:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 178450 Links: 1 2024-01-16T14:53:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-16 14:53:41.960414986 +0000 2024-01-16T14:53:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-16 14:50:37.071784252 +0000 2024-01-16T14:53:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-16 14:50:37.071784252 +0000 2024-01-16T14:53:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-01-16T14:53:52.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-16T14:53:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-16T14:53:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-16T14:53:53.015 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000113192 s, 4.5 MB/s 2024-01-16T14:53:53.017 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-16T14:53:53.080 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 177620 Links: 1 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:53.142 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-16 14:53:41.960414986 +0000 2024-01-16T14:53:53.143 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-16 14:50:37.427778399 +0000 2024-01-16T14:53:53.143 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-16 14:50:37.427778399 +0000 2024-01-16T14:53:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-01-16T14:53:53.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-16T14:53:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-16T14:53:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-16T14:53:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000123373 s, 4.2 MB/s 2024-01-16T14:53:53.216 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-16T14:53:53.279 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:53.279 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:53.280 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2024-01-16T14:53:53.300 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-16T14:53:53.300 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2024-01-16T14:53:53.361 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-16T14:53:53.361 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 172817 Links: 1 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2024-01-16 14:53:33.987049814 +0000 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2024-01-16 14:49:52.679260215 +0000 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2024-01-16 14:49:52.679260215 +0000 2024-01-16T14:53:53.362 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2024-01-16T14:53:53.362 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-16T14:53:53.433 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2024-01-16T14:53:53.433 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2024-01-16T14:53:53.433 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000245911 s, 2.1 MB/s 2024-01-16T14:53:53.434 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-16T14:53:53.497 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2024-01-16T14:53:53.558 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 174678 Links: 1 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2024-01-16 14:53:33.987049814 +0000 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2024-01-16 14:49:53.007253189 +0000 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2024-01-16 14:49:53.007253189 +0000 2024-01-16T14:53:53.559 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2024-01-16T14:53:53.559 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-16T14:53:53.630 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2024-01-16T14:53:53.630 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2024-01-16T14:53:53.630 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000261724 s, 2.0 MB/s 2024-01-16T14:53:53.631 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-16T14:53:53.693 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 172860 Links: 1 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:53.754 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2024-01-16 14:53:33.987049814 +0000 2024-01-16T14:53:53.755 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2024-01-16 14:49:53.345245949 +0000 2024-01-16T14:53:53.755 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2024-01-16 14:49:53.345245949 +0000 2024-01-16T14:53:53.755 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2024-01-16T14:53:53.755 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-16T14:53:53.825 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2024-01-16T14:53:53.825 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2024-01-16T14:53:53.825 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000193359 s, 2.6 MB/s 2024-01-16T14:53:53.827 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-16T14:53:53.889 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2024-01-16T14:53:53.950 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-16T14:53:53.950 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 176534 Links: 1 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2024-01-16 14:53:33.987049814 +0000 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2024-01-16 14:49:53.705238238 +0000 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2024-01-16 14:49:53.705238238 +0000 2024-01-16T14:53:53.951 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2024-01-16T14:53:53.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-16T14:53:54.021 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2024-01-16T14:53:54.021 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2024-01-16T14:53:54.021 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000237415 s, 2.2 MB/s 2024-01-16T14:53:54.023 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-16T14:53:54.085 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:54.085 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi093.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-16T14:53:54.085 INFO:tasks.ceph:Generating config... 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] debug client = 20 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] debug ms = 1 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] pid file = 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-16T14:53:54.087 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-16T14:53:54.087 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-16T14:53:54.087 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-16T14:53:54.087 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-16T14:53:54.087 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-16T14:53:54.087 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-16T14:53:54.087 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mgr] debug client = 20 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-16T14:53:54.088 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-16T14:53:54.088 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-16T14:53:54.089 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-16T14:53:54.090 INFO:tasks.ceph:Setting up mon.a... 2024-01-16T14:53:54.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-16T14:53:54.161 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-01-16T14:53:54.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-16T14:53:54.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:53:54.259 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.93'), ('mon.c', '[v2:172.21.15.93:3301,v1:172.21.15.93:6790]')] 2024-01-16T14:53:54.259 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-16T14:53:54.260 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:54.260 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-16T14:53:54.320 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.93 --addv c '[v2:172.21.15.93:3301,v1:172.21.15.93:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:53:54.420 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:53:54.420 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid d39fe2d6-b323-4e7f-9f15-004380a58c0d 2024-01-16T14:53:54.420 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = pacific 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:fsid d39fe2d6-b323-4e7f-9f15-004380a58c0d 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-01-16T14:53:54.437689+0000 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-01-16T14:53:54.437689+0000 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 16 (pacific) 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon.b 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] mon.c 2024-01-16T14:53:54.421 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-16T14:53:54.422 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-16T14:53:54.441 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d39fe2d6-b323-4e7f-9f15-004380a58c0d... 2024-01-16T14:53:54.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-16T14:53:54.484 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-16T14:53:54.538 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-01-16T14:53:54.538 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.539 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-01-16T14:53:54.540 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on skipped message = False 2024-01-16T14:53:54.541 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = d39fe2d6-b323-4e7f-9f15-004380a58c0d 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.542 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2024-01-16T14:53:54.543 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = bitmap 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 1/20 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 1/20 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 4/10 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-01-16T14:53:54.544 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug finisher = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-01-16T14:53:54.545 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-01-16T14:53:54.546 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-16T14:53:54.547 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = "" 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2024-01-16T14:53:54.548 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2024-01-16T14:53:54.549 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-16T14:53:54.550 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2024-01-16T14:53:54.551 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.26,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on skipped message = False 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = d39fe2d6-b323-4e7f-9f15-004380a58c0d 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.552 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2024-01-16T14:53:54.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2024-01-16T14:53:54.554 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2024-01-16T14:53:54.555 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug finisher = 20 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2024-01-16T14:53:54.556 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2024-01-16T14:53:54.557 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-16T14:53:54.558 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2024-01-16T14:53:54.559 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = True 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = True 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2024-01-16T14:53:54.560 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2024-01-16T14:53:54.561 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-16T14:53:54.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-16T14:53:54.787 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-16T14:53:54.788 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:54.788 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-16T14:53:54.809 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:54.809 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-16T14:53:54.871 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:53:54.871 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:54.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-16T14:53:54.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:53:55.033 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:53:55.033 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:53:55.168 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:53:55.168 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:55.168 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-16T14:53:55.168 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-16T14:53:55.214 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:53:55.214 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:53:55.275 INFO:tasks.ceph:Setting up mon nodes... 2024-01-16T14:53:55.275 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-16T14:53:55.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-16T14:53:55.344 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-16T14:53:55.347 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-16T14:53:55.416 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-16T14:53:55.419 INFO:tasks.ceph:Setting up mds nodes... 2024-01-16T14:53:55.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-16T14:53:55.489 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-16T14:53:55.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-16T14:53:55.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-16T14:53:55.632 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-16T14:53:55.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-16T14:53:55.665 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-16T14:53:55.732 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-16T14:53:55.735 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-16T14:53:55.764 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-16T14:53:55.863 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-16T14:53:55.865 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-16T14:53:55.892 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-16T14:53:55.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-16T14:53:55.948 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-16T14:53:55.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-16T14:53:56.014 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-16T14:53:56.030 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-16T14:53:56.074 INFO:teuthology.orchestra.run.smithi093.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-16T14:53:56.087 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-16T14:53:56.130 INFO:teuthology.orchestra.run.smithi093.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-16T14:53:56.143 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-16T14:53:56.143 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi093.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-16T14:53:56.143 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-16T14:53:56.179 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:56.180 INFO:tasks.ceph:role: osd.0 2024-01-16T14:53:56.180 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:53:56.180 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-16T14:53:56.252 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:53:56.253 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:53:56.275 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-16T14:53:56.317 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-16T14:53:56.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-16T14:53:56.407 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-16T14:53:56.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-16T14:53:56.519 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:56.520 INFO:tasks.ceph:role: osd.1 2024-01-16T14:53:56.520 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:53:56.520 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-16T14:53:56.595 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:53:56.595 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:53:56.596 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:53:56.597 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:53:56.618 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-16T14:53:56.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-16T14:53:56.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-16T14:53:56.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-16T14:53:56.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-16T14:53:56.821 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:56.821 INFO:tasks.ceph:role: osd.2 2024-01-16T14:53:56.821 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:53:56.821 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-16T14:53:56.895 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:53:56.895 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:53:56.896 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:53:56.918 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-16T14:53:56.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-16T14:53:56.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-16T14:53:57.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-16T14:53:57.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-16T14:53:57.124 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-16T14:53:57.124 INFO:tasks.ceph:role: osd.3 2024-01-16T14:53:57.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:53:57.124 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:53:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:53:57.222 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-16T14:53:57.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-16T14:53:57.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-16T14:53:57.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-16T14:53:57.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:53:57.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.466+0000 7f4e7d871640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-16T14:53:57.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.466+0000 7f4e7d871640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-16T14:53:57.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.466+0000 7f4e7d871640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:53:57.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.467+0000 7f4e7d871640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:53:57.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.467+0000 7f4e7d871640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-16T14:53:57.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:53:57.467+0000 7f4e7d871640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-16T14:54:00.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-16T14:54:00.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.455+0000 7f5e5777b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-16T14:54:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.455+0000 7f5e5777b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-16T14:54:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.455+0000 7f5e5777b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:54:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.455+0000 7f5e5777b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:54:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.455+0000 7f5e5777b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-16T14:54:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:00.456+0000 7f5e5777b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-16T14:54:03.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-16T14:54:03.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.441+0000 7f8336eeb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.441+0000 7f8336eeb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.441+0000 7f8336eeb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.441+0000 7f8336eeb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.441+0000 7f8336eeb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-16T14:54:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:03.442+0000 7f8336eeb640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-16T14:54:06.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-16T14:54:06.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-16T14:54:06.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:06.423+0000 7fa5be129640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-16T14:54:09.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-16T14:54:09.296 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-16T14:54:09.329 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:54:09.329 INFO:tasks.ceph:role: osd.4 2024-01-16T14:54:09.330 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:54:09.330 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:54:09.408 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:54:09.409 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:54:09.409 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:54:09.838 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2024-01-16T14:54:09.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2024-01-16T14:54:09.840 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-16T14:54:10.037 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-16T14:54:10.076 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-16T14:54:10.145 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:54:10.145 INFO:tasks.ceph:role: osd.5 2024-01-16T14:54:10.145 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:54:10.145 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-16T14:54:10.221 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:54:10.221 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:54:10.221 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:54:10.222 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:54:10.630 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2024-01-16T14:54:10.632 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2024-01-16T14:54:10.632 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-16T14:54:10.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-16T14:54:10.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-16T14:54:10.858 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:54:10.858 INFO:tasks.ceph:role: osd.6 2024-01-16T14:54:10.858 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:54:10.858 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-16T14:54:10.936 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:54:10.936 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:54:10.936 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:54:10.937 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:54:11.377 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2024-01-16T14:54:11.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2024-01-16T14:54:11.380 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-16T14:54:11.544 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-16T14:54:11.583 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-16T14:54:11.655 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-16T14:54:11.655 INFO:tasks.ceph:role: osd.7 2024-01-16T14:54:11.655 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:54:11.655 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-16T14:54:11.734 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-16T14:54:12.177 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2024-01-16T14:54:12.179 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2024-01-16T14:54:12.179 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-16T14:54:12.365 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-16T14:54:12.404 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:12.495 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.511+0000 7f54e1412640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-16T14:54:12.495 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.511+0000 7f54e1412640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-16T14:54:12.496 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.511+0000 7f54e1412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:54:12.496 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.511+0000 7f54e1412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:54:12.496 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.512+0000 7f54e1412640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-16T14:54:12.496 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:12.512+0000 7f54e1412640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-16T14:54:15.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-16T14:54:15.394 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:15.491 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.507+0000 7fd48cb9f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-16T14:54:15.491 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.508+0000 7fd48cb9f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-16T14:54:15.491 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.508+0000 7fd48cb9f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:54:15.491 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.508+0000 7fd48cb9f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:54:15.491 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.508+0000 7fd48cb9f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-16T14:54:15.492 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:15.508+0000 7fd48cb9f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-16T14:54:18.352 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-16T14:54:18.385 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:18.481 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.497+0000 7f525c6d0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-16T14:54:18.482 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.498+0000 7f525c6d0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-16T14:54:18.482 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.498+0000 7f525c6d0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:54:18.482 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.498+0000 7f525c6d0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:54:18.482 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.498+0000 7f525c6d0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-16T14:54:18.482 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:18.498+0000 7f525c6d0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-16T14:54:21.328 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-16T14:54:21.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.465+0000 7f6d0673e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.465+0000 7f6d0673e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.465+0000 7f6d0673e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.465+0000 7f6d0673e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.465+0000 7f6d0673e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-16T14:54:21.448 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:21.466+0000 7f6d0673e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-16T14:54:22.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:54:22.324 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:54:24.301 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-16T14:54:24.335 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-16T14:54:24.335 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-16T14:54:24.370 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.370 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-16T14:54:24.445 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-16T14:54:24.519 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-16T14:54:24.595 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-16T14:54:24.670 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-16T14:54:24.744 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:24.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-16T14:54:24.818 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:24.818 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-16T14:54:24.851 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:24.851 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-16T14:54:24.922 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:24.922 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-16T14:54:24.992 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:24.992 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-16T14:54:25.062 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:25.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-16T14:54:25.130 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:25.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-16T14:54:25.198 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:25.198 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-16T14:54:25.265 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:25.265 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-16T14:54:25.287 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:25.287 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-16T14:54:25.349 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:25.349 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-16T14:54:25.367 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:25.367 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-16T14:54:25.424 INFO:tasks.ceph:Adding keys to all mons... 2024-01-16T14:54:25.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-16T14:54:25.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZlBayOFRAA53grUUs3v0uIUgKWz5ep3Q== 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZl4XAuHhAA4D3kwEeG+gbjNF4/rF0U2Q== 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZltK62JhAArIT8N0ucUNkEhlpCFndkOA== 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCFmKZlyKfdGxAATOUoliQPcYm2vSvVcBZ9rw== 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCImKZlUTczGxAAyhDeEJ1jrpRNOqtq4YzKFw== 2024-01-16T14:54:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCLmKZlaAhbGhAAdBk7IP19PAqvO7FzA6OjHA== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCOmKZlXT5FGRAAIXAbV/zGy0mu1hML6KdHfg== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZlJLDfGRAAK1YRyL4/DNnMX0M7WJThLA== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZlIT6wLBAASm0XUVaPtvstSdGMyfPAOA== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZlDRx6NBAA+uk0U+typ/+LV+BO8LHnYg== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCUmKZlf9aQHhAAfHJuQ+qPwhvpFTO3LNDxgA== 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-01-16T14:54:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCXmKZl+MVTHhAAtNjpAJRt0UluZ0W7ulYcug== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCamKZlQAW8HRAADpM2I3F32HIjlkmeIBxU9g== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCdmKZlo+HAGxAA4nE/koKFEvlcqHZAeibWcg== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZl8JqJORAAXPhIkgN/hhjJtdjFgPeRLQ== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[client.2] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCEmKZlQ4foARAAdLlG4GlY3XsbaeZnnhsqdQ== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCEmKZlTU9/BRAA3FQuMz/iXQtANu603C0hDQ== 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout:[client.3] 2024-01-16T14:54:25.469 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCEmKZl8AnOCBAAaUevXdwBXt8tPHjFBxhu6A== 2024-01-16T14:54:25.488 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr.y] 2024-01-16T14:54:25.488 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZlBayOFRAA53grUUs3v0uIUgKWz5ep3Q== 2024-01-16T14:54:25.488 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZl4XAuHhAA4D3kwEeG+gbjNF4/rF0U2Q== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZltK62JhAArIT8N0ucUNkEhlpCFndkOA== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.0] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCFmKZlyKfdGxAATOUoliQPcYm2vSvVcBZ9rw== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.1] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCImKZlUTczGxAAyhDeEJ1jrpRNOqtq4YzKFw== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.2] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCLmKZlaAhbGhAAdBk7IP19PAqvO7FzA6OjHA== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.3] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCOmKZlXT5FGRAAIXAbV/zGy0mu1hML6KdHfg== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr.x] 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZlJLDfGRAAK1YRyL4/DNnMX0M7WJThLA== 2024-01-16T14:54:25.489 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZlIT6wLBAASm0XUVaPtvstSdGMyfPAOA== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZlDRx6NBAA+uk0U+typ/+LV+BO8LHnYg== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.4] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCUmKZlf9aQHhAAfHJuQ+qPwhvpFTO3LNDxgA== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.5] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCXmKZl+MVTHhAAtNjpAJRt0UluZ0W7ulYcug== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.6] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCamKZlQAW8HRAADpM2I3F32HIjlkmeIBxU9g== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.7] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCdmKZlo+HAGxAA4nE/koKFEvlcqHZAeibWcg== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[client.0] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCDmKZl8JqJORAAXPhIkgN/hhjJtdjFgPeRLQ== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[client.2] 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCEmKZlQ4foARAAdLlG4GlY3XsbaeZnnhsqdQ== 2024-01-16T14:54:25.490 INFO:teuthology.orchestra.run.smithi093.stdout:[client.1] 2024-01-16T14:54:25.491 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCEmKZlTU9/BRAA3FQuMz/iXQtANu603C0hDQ== 2024-01-16T14:54:25.491 INFO:teuthology.orchestra.run.smithi093.stdout:[client.3] 2024-01-16T14:54:25.491 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCEmKZl8AnOCBAAaUevXdwBXt8tPHjFBxhu6A== 2024-01-16T14:54:25.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:54:25.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:54:25.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:25.588 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:25.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:25.654 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:25.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.719 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.839 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.898 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:25.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:54:25.959 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-16T14:54:26.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:26.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:26.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:26.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-16T14:54:26.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.145 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.265 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.324 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-16T14:54:26.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.384 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.447 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.511 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.575 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-16T14:54:26.636 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-16T14:54:26.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-16T14:54:26.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:54:26.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-16T14:54:26.807 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-16T14:54:26.835 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:54:26.934 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-16T14:54:26.961 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-16T14:54:27.027 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-16T14:54:27.127 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-16T14:54:27.153 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:27.156 DEBUG:teuthology.orchestra.run.smithi093:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-16T14:54:27.210 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-16T14:54:27.210 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-16T14:54:27.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-16T14:54:27.218 INFO:tasks.ceph.mon.a:Started 2024-01-16T14:54:27.218 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-16T14:54:27.218 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-16T14:54:27.252 INFO:tasks.ceph.mon.b:Started 2024-01-16T14:54:27.252 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-16T14:54:27.252 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-16T14:54:27.254 INFO:tasks.ceph.mon.c:Started 2024-01-16T14:54:27.254 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-16T14:54:27.254 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-16T14:54:27.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-16T14:54:27.257 INFO:tasks.ceph.mgr.y:Started 2024-01-16T14:54:27.257 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-16T14:54:27.257 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-16T14:54:27.259 INFO:tasks.ceph.mgr.x:Started 2024-01-16T14:54:27.259 INFO:tasks.ceph:Setting crush tunables to default 2024-01-16T14:54:27.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-01-16T14:54:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.415+0000 7fdf6878a700 1 Processor -- start 2024-01-16T14:54:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.415+0000 7fdf6878a700 1 -- start start 2024-01-16T14:54:27.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf6878a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf6878a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf6878a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600cfd60 con 0x7fdf600d0ae0 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf6878a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600cf4b0 con 0x7fdf6015e000 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf6878a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6006f430 con 0x7fdf6006fe00 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54026/0 (socket says 172.21.15.26:54026) 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 learned_addr learned my addr 172.21.15.26:0/1768458681 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf65d25700 1 --1- 172.21.15.26:0/1768458681 >> v1:172.21.15.26:6789/0 conn(0x7fdf600d0ae0 0x7fdf6006f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60724/0 (socket says 172.21.15.26:60724) 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 msgr2=0x7fdf6006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 msgr2=0x7fdf6006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66526700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66d27700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66d27700 1 -- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 msgr2=0x7fdf600cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-16T14:54:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66d27700 1 -- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 msgr2=0x7fdf600cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-16T14:54:27.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66d27700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-16T14:54:27.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.416+0000 7fdf66d27700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-16T14:54:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.616+0000 7fdf65524700 1 -- 172.21.15.26:0/1768458681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028059011 0 0) 0x7fdf600cfd60 con 0x7fdf600d0ae0 2024-01-16T14:54:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.616+0000 7fdf65524700 1 -- 172.21.15.26:0/1768458681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf4c0035b0 con 0x7fdf600d0ae0 2024-01-16T14:54:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66526700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.26:3300/0 conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66d27700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 msgr2=0x7fdf600cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66526700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 >> v1:172.21.15.26:6789/0 conn(0x7fdf600d0ae0 legacy=0x7fdf6006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66526700 1 -- 172.21.15.26:0/1768458681 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600cf1d0 con 0x7fdf6015e000 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.617+0000 7fdf66d27700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.618+0000 7fdf66526700 1 --2- 172.21.15.26:0/1768458681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf6006eef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf600ce1f0 tx=0x7fdf5c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c157a5f1bd6a9c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.618+0000 7fdf65524700 1 -- 172.21.15.26:0/1768458681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c004230 con 0x7fdf6015e000 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.618+0000 7fdf65524700 1 -- 172.21.15.26:0/1768458681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c004390 con 0x7fdf6015e000 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.618+0000 7fdf65524700 1 -- 172.21.15.26:0/1768458681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0036a0 con 0x7fdf6015e000 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 -- 172.21.15.26:0/1768458681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 msgr2=0x7fdf6006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 --2- 172.21.15.26:0/1768458681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf6006eef0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf600ce1f0 tx=0x7fdf5c009c30 comp rx=0 tx=0).stop 2024-01-16T14:54:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 -- 172.21.15.26:0/1768458681 shutdown_connections 2024-01-16T14:54:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 --2- 172.21.15.26:0/1768458681 >> v2:172.21.15.93:3300/0 conn(0x7fdf6006fe00 0x7fdf600cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 --2- 172.21.15.26:0/1768458681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf6006eef0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 -- 172.21.15.26:0/1768458681 >> 172.21.15.26:0/1768458681 conn(0x7fdf600b3720 msgr2=0x7fdf600b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 -- 172.21.15.26:0/1768458681 shutdown_connections 2024-01-16T14:54:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.619+0000 7fdf6878a700 1 -- 172.21.15.26:0/1768458681 wait complete. 2024-01-16T14:54:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.621+0000 7fdf6878a700 1 Processor -- start 2024-01-16T14:54:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.621+0000 7fdf6878a700 1 -- start start 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf6006fe00 0x7fdf6015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf600c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c85e0 con 0x7fdf6015e000 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c59a0 con 0x7fdf600d0ae0 2024-01-16T14:54:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.622+0000 7fdf6878a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c7a60 con 0x7fdf6006fe00 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf66d27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf600c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:36964/0 (socket says 172.21.15.26:36964) 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 -- 172.21.15.26:0/4154087743 learned_addr learned my addr 172.21.15.26:0/4154087743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 -- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf6006fe00 msgr2=0x7fdf6015bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf6006fe00 0x7fdf6015bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 -- 172.21.15.26:0/4154087743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 msgr2=0x7fdf600c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf600c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 -- 172.21.15.26:0/4154087743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58009ad0 con 0x7fdf600d0ae0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf65d25700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c410 tx=0x7fdf5000c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005cb0 con 0x7fdf600d0ae0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50005e10 con 0x7fdf600d0ae0 2024-01-16T14:54:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.623+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0095b0 con 0x7fdf600d0ae0 2024-01-16T14:54:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.624+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50007310 con 0x7fdf600d0ae0 2024-01-16T14:54:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.624+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf600c8020 con 0x7fdf600d0ae0 2024-01-16T14:54:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.624+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdf5001c440 con 0x7fdf600d0ae0 2024-01-16T14:54:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.624+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf50012740 con 0x7fdf600d0ae0 2024-01-16T14:54:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.625+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf60075480 con 0x7fdf600d0ae0 2024-01-16T14:54:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.627+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdf50017070 con 0x7fdf600d0ae0 2024-01-16T14:54:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.714+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdf600b7b50 con 0x7fdf600d0ae0 2024-01-16T14:54:27.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.718+0000 7fdf56ffd700 1 -- 172.21.15.26:0/4154087743 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdf5001c920 con 0x7fdf600d0ae0 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 msgr2=0x7fdf600c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000c410 tx=0x7fdf5000c720 comp rx=0 tx=0).stop 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 shutdown_connections 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf6015e000 0x7fdf600c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf6006fe00 0x7fdf6015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 --2- 172.21.15.26:0/4154087743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf600d0ae0 0x7fdf600c6f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 >> 172.21.15.26:0/4154087743 conn(0x7fdf600b3720 msgr2=0x7fdf600c4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 shutdown_connections 2024-01-16T14:54:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:27.719+0000 7fdf6878a700 1 -- 172.21.15.26:0/4154087743 wait complete. 2024-01-16T14:54:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-01-16T14:54:27.716 INFO:tasks.ceph:check_enable_crimson: False 2024-01-16T14:54:27.716 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-16T14:54:27.717 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:27.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-16T14:54:27.748 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:27.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-16T14:54:27.820 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:27.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-16T14:54:27.889 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:54:27.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-16T14:54:27.958 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:27.958 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-16T14:54:27.984 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:27.984 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-16T14:54:28.020 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.036+0000 7f82bec3f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-16T14:54:28.045 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:28.045 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-16T14:54:28.081 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.098+0000 7f9c9b6b0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-16T14:54:28.102 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.119+0000 7f82bec3f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-16T14:54:28.111 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:54:28.111 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-16T14:54:28.172 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.189+0000 7f9c9b6b0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-16T14:54:28.179 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 70a24a3c-be2b-4420-855a-5740cd033b47 0 2024-01-16T14:54:28.181 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.198+0000 7f82bec3f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-16T14:54:28.255 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.272+0000 7f9c9b6b0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-16T14:54:28.259 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.275+0000 7f82bec3f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-16T14:54:28.306 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.323+0000 7fd6e8562700 1 Processor -- start 2024-01-16T14:54:28.307 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.323+0000 7fd6e8562700 1 -- start start 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e8562700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e8562700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e8562700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00cefa0 con 0x7fd6e015c2b0 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e8562700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e006fa90 con 0x7fd6e006afd0 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e8562700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e006b8f0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e62fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e6aff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e5afd700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fd6e015c2b0 0x7fd6e006e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40214/0 (socket says 172.21.15.93:40214) 2024-01-16T14:54:28.309 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e5afd700 1 -- 172.21.15.93:0/1401926662 learned_addr learned my addr 172.21.15.93:0/1401926662 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e62fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44332/0 (socket says 172.21.15.93:44332) 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e6aff700 1 -- 172.21.15.93:0/1401926662 >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 msgr2=0x7fd6e006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e6aff700 1 --2- 172.21.15.93:0/1401926662 >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e6aff700 1 -- 172.21.15.93:0/1401926662 >> v1:172.21.15.93:6789/0 conn(0x7fd6e015c2b0 legacy=0x7fd6e006e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e6aff700 1 -- 172.21.15.93:0/1401926662 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e006f7b0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.324+0000 7fd6e62fe700 1 --2- 172.21.15.93:0/1401926662 >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e6aff700 1 --2- 172.21.15.93:0/1401926662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00d0140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4044e799d66fc764 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e52fc700 1 -- 172.21.15.93:0/1401926662 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800e070 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e52fc700 1 -- 172.21.15.93:0/1401926662 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d800bb20 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e52fc700 1 -- 172.21.15.93:0/1401926662 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8012430 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 -- 172.21.15.93:0/1401926662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 msgr2=0x7fd6e00d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 --2- 172.21.15.93:0/1401926662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00d0140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 -- 172.21.15.93:0/1401926662 shutdown_connections 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 --2- 172.21.15.93:0/1401926662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00d0140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.310 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 --2- 172.21.15.93:0/1401926662 >> v2:172.21.15.93:3300/0 conn(0x7fd6e006afd0 0x7fd6e006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.311 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 -- 172.21.15.93:0/1401926662 >> 172.21.15.93:0/1401926662 conn(0x7fd6e00b27d0 msgr2=0x7fd6e00b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.311 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 -- 172.21.15.93:0/1401926662 shutdown_connections 2024-01-16T14:54:28.311 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.325+0000 7fd6e8562700 1 -- 172.21.15.93:0/1401926662 wait complete. 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 Processor -- start 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 -- start start 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e006afd0 0x7fd6e015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 0x7fd6e00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013ded0 con 0x7fd6e015c2b0 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013ea10 con 0x7fd6e006afd0 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e8562700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013ecf0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e62fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e006afd0 0x7fd6e015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.313 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e5afd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e5afd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44546/0 (socket says 172.21.15.93:44546) 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.327+0000 7fd6e5afd700 1 -- 172.21.15.93:0/668830048 learned_addr learned my addr 172.21.15.93:0/668830048 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e6aff700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 0x7fd6e00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 -- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e006afd0 msgr2=0x7fd6e015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e006afd0 0x7fd6e015aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 -- 172.21.15.93:0/668830048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 msgr2=0x7fd6e00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 0x7fd6e00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 -- 172.21.15.93:0/668830048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0094a0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e6aff700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 0x7fd6e00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e5afd700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c480 tx=0x7fd6d000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0005d70 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d00070d0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0007440 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d8009a70 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e013e3c0 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.314 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd6d001c440 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.315 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.328+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd6d0012740 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.315 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.329+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0075480 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.315 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.332+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd6d0021060 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.335 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.352+0000 7f9c9b6b0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-16T14:54:28.400 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.416+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "70a24a3c-be2b-4420-855a-5740cd033b47", "id": 0} v 0) v1 -- 0x7fd6e00ccc50 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.403 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.420+0000 7fd6d6ffd700 1 -- 172.21.15.93:0/668830048 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70a24a3c-be2b-4420-855a-5740cd033b47", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd6d0017070 con 0x7fd6e00cfcb0 2024-01-16T14:54:28.403 INFO:teuthology.orchestra.run.smithi093.stdout:0 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 msgr2=0x7fd6e00b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c480 tx=0x7fd6d000c790 comp rx=0 tx=0).stop 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 shutdown_connections 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6e015c2b0 0x7fd6e00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e006afd0 0x7fd6e015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.404 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 --2- 172.21.15.93:0/668830048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e00cfcb0 0x7fd6e00b0ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 >> 172.21.15.93:0/668830048 conn(0x7fd6e00b27d0 msgr2=0x7fd6e00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 shutdown_connections 2024-01-16T14:54:28.405 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.421+0000 7fd6e8562700 1 -- 172.21.15.93:0/668830048 wait complete. 2024-01-16T14:54:28.416 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 6b5503f8-087d-41d5-a7a9-c63e02c01c36 1 2024-01-16T14:54:28.427 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.443+0000 7f82bec3f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:54:28.517 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.534+0000 7f9c9b6b0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:54:28.537 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.554+0000 7ff08ef47700 1 Processor -- start 2024-01-16T14:54:28.537 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.554+0000 7ff08ef47700 1 -- start start 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08ef47700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08ef47700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff088145650 0x7ff08806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08ef47700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08806eb80 con 0x7ff088145100 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08ef47700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08806f500 con 0x7ff088145650 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08ef47700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880cd610 con 0x7ff088144c10 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08cce3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff088145650 0x7ff08806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08cce3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44548/0 (socket says 172.21.15.93:44548) 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff088145650 0x7ff08806b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44336/0 (socket says 172.21.15.93:44336) 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08cce3700 1 -- 172.21.15.93:0/1184199505 learned_addr learned my addr 172.21.15.93:0/1184199505 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 -- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 msgr2=0x7ff08806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 -- 172.21.15.93:0/1184199505 >> v1:172.21.15.26:6789/0 conn(0x7ff088145100 legacy=0x7ff088071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.540 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 -- 172.21.15.93:0/1184199505 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088160720 con 0x7ff088145650 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08cce3700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff08d4e4700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088145650 0x7ff08806b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff080009d90 tx=0x7ff08000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89db9d7c3b1452cb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff0877fe700 1 -- 172.21.15.93:0/1184199505 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000e070 con 0x7ff088145650 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff0877fe700 1 -- 172.21.15.93:0/1184199505 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000bb20 con 0x7ff088145650 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.555+0000 7ff0877fe700 1 -- 172.21.15.93:0/1184199505 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080012430 con 0x7ff088145650 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 -- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088145650 msgr2=0x7ff08806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088145650 0x7ff08806b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff080009d90 tx=0x7ff08000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 -- 172.21.15.93:0/1184199505 shutdown_connections 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088145650 0x7ff08806b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 --2- 172.21.15.93:0/1184199505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088144c10 0x7ff08806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 -- 172.21.15.93:0/1184199505 >> 172.21.15.93:0/1184199505 conn(0x7ff0880b2740 msgr2=0x7ff0880b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 -- 172.21.15.93:0/1184199505 shutdown_connections 2024-01-16T14:54:28.541 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.556+0000 7ff08ef47700 1 -- 172.21.15.93:0/1184199505 wait complete. 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.557+0000 7ff08ef47700 1 Processor -- start 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.557+0000 7ff08ef47700 1 -- start start 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 0x7ff08815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff088145100 0x7ff0880afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 0x7ff0881409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880aed70 con 0x7ff088145100 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880adcd0 con 0x7ff088144c10 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08ef47700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880adfb0 con 0x7ff088145650 2024-01-16T14:54:28.544 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08cce3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 0x7ff08815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08d4e4700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 0x7ff0881409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08cce3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 0x7ff08815b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44344/0 (socket says 172.21.15.93:44344) 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff08cce3700 1 -- 172.21.15.93:0/492818738 learned_addr learned my addr 172.21.15.93:0/492818738 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.558+0000 7ff087fff700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff088145100 0x7ff0880afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 -- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 msgr2=0x7ff08815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 0x7ff08815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 -- 172.21.15.93:0/492818738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff088145100 msgr2=0x7ff0880afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff088145100 0x7ff0880afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 -- 172.21.15.93:0/492818738 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009550 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08d4e4700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 0x7ff0881409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff080016000 tx=0x7ff080004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080004800 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff080020070 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff080009a70 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0800276c0 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0880ae570 con 0x7ff088145650 2024-01-16T14:54:28.545 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff080027b20 con 0x7ff088145650 2024-01-16T14:54:28.546 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.559+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff080022070 con 0x7ff088145650 2024-01-16T14:54:28.546 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.560+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff088075480 con 0x7ff088145650 2024-01-16T14:54:28.546 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.562+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff08002c060 con 0x7ff088145650 2024-01-16T14:54:28.586 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.603+0000 7f82bec3f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-16T14:54:28.631 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.648+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6b5503f8-087d-41d5-a7a9-c63e02c01c36", "id": 1} v 0) v1 -- 0x7ff0880d0840 con 0x7ff088145650 2024-01-16T14:54:28.634 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.651+0000 7ff0857fa700 1 -- 172.21.15.93:0/492818738 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b5503f8-087d-41d5-a7a9-c63e02c01c36", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff0800128c0 con 0x7ff088145650 2024-01-16T14:54:28.634 INFO:teuthology.orchestra.run.smithi093.stdout:1 2024-01-16T14:54:28.635 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 msgr2=0x7ff0881409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 0x7ff0881409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff080016000 tx=0x7ff080004ad0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 shutdown_connections 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff088145650 0x7ff0881409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff088144c10 0x7ff08815b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 --2- 172.21.15.93:0/492818738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff088145100 0x7ff0880afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 >> 172.21.15.93:0/492818738 conn(0x7ff0880b2740 msgr2=0x7ff0880b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 shutdown_connections 2024-01-16T14:54:28.636 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.652+0000 7ff08ef47700 1 -- 172.21.15.93:0/492818738 wait complete. 2024-01-16T14:54:28.647 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 341f4396-d5d7-4489-9f9d-16991e80028b 2 2024-01-16T14:54:28.691 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.708+0000 7f9c9b6b0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-16T14:54:28.704 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.720+0000 7f82bec3f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-16T14:54:28.803 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.820+0000 7f82bec3f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-16T14:54:28.809 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.825+0000 7fd4205c6700 1 Processor -- start 2024-01-16T14:54:28.809 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.825+0000 7fd4205c6700 1 -- start start 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd4205c6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd41815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd4205c6700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd4180ce5c0 0x7fd4180cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd4205c6700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180d0a90 con 0x7fd4180cf2b0 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd4205c6700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180cef90 con 0x7fd4180ce5c0 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd4205c6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41806ee10 con 0x7fd41815bde0 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd41eb63700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd4180ce5c0 0x7fd4180cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.813 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd41db61700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fd4180cf2b0 0x7fd4180d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40222/0 (socket says 172.21.15.93:40222) 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd41e362700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd41815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.826+0000 7fd41db61700 1 -- 172.21.15.93:0/4283747610 learned_addr learned my addr 172.21.15.93:0/4283747610 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41d360700 1 -- 172.21.15.93:0/4283747610 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22379430 0 0) 0x7fd4180d0a90 con 0x7fd4180cf2b0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41d360700 1 -- 172.21.15.93:0/4283747610 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4040035b0 con 0x7fd4180cf2b0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41eb63700 1 -- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 msgr2=0x7fd41815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41eb63700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd41815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41eb63700 1 -- 172.21.15.93:0/4283747610 >> v1:172.21.15.93:6789/0 conn(0x7fd4180cf2b0 legacy=0x7fd4180d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41eb63700 1 -- 172.21.15.93:0/4283747610 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41806f0f0 con 0x7fd4180ce5c0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41e362700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd41815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41eb63700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180cea50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd410009d90 tx=0x7fd41000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e6f8385bcb1e86 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41d360700 1 -- 172.21.15.93:0/4283747610 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000e070 con 0x7fd4180ce5c0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41d360700 1 -- 172.21.15.93:0/4283747610 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000bb20 con 0x7fd4180ce5c0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.827+0000 7fd41d360700 1 -- 172.21.15.93:0/4283747610 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410012430 con 0x7fd4180ce5c0 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 -- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 msgr2=0x7fd4180cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180cea50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd410009d90 tx=0x7fd41000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:28.814 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 -- 172.21.15.93:0/4283747610 shutdown_connections 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180cea50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 --2- 172.21.15.93:0/4283747610 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd41815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 -- 172.21.15.93:0/4283747610 >> 172.21.15.93:0/4283747610 conn(0x7fd4180b3720 msgr2=0x7fd4180b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 -- 172.21.15.93:0/4283747610 shutdown_connections 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.828+0000 7fd4205c6700 1 -- 172.21.15.93:0/4283747610 wait complete. 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.829+0000 7fd4205c6700 1 Processor -- start 2024-01-16T14:54:28.815 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.829+0000 7fd4205c6700 1 -- start start 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180cf2b0 0x7fd4180cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418150b50 con 0x7fd4180cf2b0 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180b6770 con 0x7fd4180ce5c0 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd4205c6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180c8dd0 con 0x7fd41815bde0 2024-01-16T14:54:28.817 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd41eb63700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd41e362700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd41e362700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44356/0 (socket says 172.21.15.93:44356) 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd41eb63700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44566/0 (socket says 172.21.15.93:44566) 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.830+0000 7fd41e362700 1 -- 172.21.15.93:0/24504859 learned_addr learned my addr 172.21.15.93:0/24504859 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41db61700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180cf2b0 0x7fd4180cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 -- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 msgr2=0x7fd4181f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 -- 172.21.15.93:0/24504859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180cf2b0 msgr2=0x7fd4180cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180cf2b0 0x7fd4180cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 -- 172.21.15.93:0/24504859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140094a0 con 0x7fd4180ce5c0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41eb63700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd41e362700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ebe0 tx=0x7fd41400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400c850 con 0x7fd4180ce5c0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41400c9b0 con 0x7fd4180ce5c0 2024-01-16T14:54:28.818 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400cd40 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd410009a70 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4180b61b0 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd414005d20 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.831+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd414006840 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.832+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4180c9100 con 0x7fd4180ce5c0 2024-01-16T14:54:28.819 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.834+0000 7f9c9b6b0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-16T14:54:28.819 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.834+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd414021060 con 0x7fd4180ce5c0 2024-01-16T14:54:28.883 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:28.899+0000 7f82bec3f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-16T14:54:28.903 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.920+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "341f4396-d5d7-4489-9f9d-16991e80028b", "id": 2} v 0) v1 -- 0x7fd418075480 con 0x7fd4180ce5c0 2024-01-16T14:54:28.906 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.923+0000 7fd40effd700 1 -- 172.21.15.93:0/24504859 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "341f4396-d5d7-4489-9f9d-16991e80028b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd414017070 con 0x7fd4180ce5c0 2024-01-16T14:54:28.906 INFO:teuthology.orchestra.run.smithi093.stdout:2 2024-01-16T14:54:28.907 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 msgr2=0x7fd4180b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ebe0 tx=0x7fd41400eef0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 shutdown_connections 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd41815bde0 0x7fd4181f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4180ce5c0 0x7fd4180b56a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 --2- 172.21.15.93:0/24504859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4180cf2b0 0x7fd4180cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 >> 172.21.15.93:0/24504859 conn(0x7fd4180b3720 msgr2=0x7fd4180b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 shutdown_connections 2024-01-16T14:54:28.908 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:28.924+0000 7fd4205c6700 1 -- 172.21.15.93:0/24504859 wait complete. 2024-01-16T14:54:28.919 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new b35b6724-3ca8-497d-b775-38a3d40791a1 3 2024-01-16T14:54:28.925 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:28.942+0000 7f9c9b6b0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-16T14:54:29.011 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:29.028+0000 7f9c9b6b0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-16T14:54:29.106 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.122+0000 7fb3181cc700 1 Processor -- start 2024-01-16T14:54:29.106 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 -- start start 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb31006b620 0x7fb3100b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb310071f50 con 0x7fb3100b5ab0 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb310072230 con 0x7fb31006b620 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.123+0000 7fb3181cc700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb310072510 con 0x7fb31006ee60 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb316769700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb31006b620 0x7fb3100b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb316769700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44580/0 (socket says 172.21.15.93:44580) 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315767700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fb3100b5ab0 0x7fb310161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40238/0 (socket says 172.21.15.93:40238) 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb316769700 1 -- 172.21.15.93:0/3177324133 learned_addr learned my addr 172.21.15.93:0/3177324133 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 -- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 msgr2=0x7fb3100b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.108 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 -- 172.21.15.93:0/3177324133 >> v1:172.21.15.93:6789/0 conn(0x7fb3100b5ab0 legacy=0x7fb310161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 -- 172.21.15.93:0/3177324133 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100727f0 con 0x7fb31006b620 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb316769700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb315f68700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006b620 0x7fb3100b3a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009840 tx=0x7fb30c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6cc445f3e0ef18 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb314f66700 1 -- 172.21.15.93:0/3177324133 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c011830 con 0x7fb31006b620 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb314f66700 1 -- 172.21.15.93:0/3177324133 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c011990 con 0x7fb31006b620 2024-01-16T14:54:29.109 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.124+0000 7fb314f66700 1 -- 172.21.15.93:0/3177324133 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c011d20 con 0x7fb31006b620 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 -- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006b620 msgr2=0x7fb3100b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006b620 0x7fb3100b3a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009840 tx=0x7fb30c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 -- 172.21.15.93:0/3177324133 shutdown_connections 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb31006ee60 0x7fb3100b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 --2- 172.21.15.93:0/3177324133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006b620 0x7fb3100b3a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 -- 172.21.15.93:0/3177324133 >> 172.21.15.93:0/3177324133 conn(0x7fb3100b2740 msgr2=0x7fb3100b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 -- 172.21.15.93:0/3177324133 shutdown_connections 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.125+0000 7fb3181cc700 1 -- 172.21.15.93:0/3177324133 wait complete. 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.126+0000 7fb3181cc700 1 Processor -- start 2024-01-16T14:54:29.110 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.126+0000 7fb3181cc700 1 -- start start 2024-01-16T14:54:29.112 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 0x7fb310136180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.112 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3100b5ab0 0x7fb3100c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310136c70 con 0x7fb31006b620 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c5d90 con 0x7fb31006ee60 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb3181cc700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c6070 con 0x7fb3100b5ab0 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44372/0 (socket says 172.21.15.93:44372) 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 -- 172.21.15.93:0/2600813840 learned_addr learned my addr 172.21.15.93:0/2600813840 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb316769700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3100b5ab0 0x7fb3100c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315f68700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 0x7fb310136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 -- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3100b5ab0 msgr2=0x7fb3100c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3100b5ab0 0x7fb3100c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 -- 172.21.15.93:0/2600813840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 msgr2=0x7fb310136180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 0x7fb310136180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315767700 1 -- 172.21.15.93:0/2600813840 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308009b30 con 0x7fb31006ee60 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.127+0000 7fb315f68700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 0x7fb310136180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:29.113 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb315767700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c4d0 tx=0x7fb30000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300005cb0 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb300005e10 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000072d0 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30c0094a0 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3100c6630 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb30001c440 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.128+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb30001cb80 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.129+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb310075480 con 0x7fb31006ee60 2024-01-16T14:54:29.114 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.131+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb300021060 con 0x7fb31006ee60 2024-01-16T14:54:29.133 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:29.150+0000 7f82bec3f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-16T14:54:29.222 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:29.239+0000 7f82bec3f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-16T14:54:29.231 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.248+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b35b6724-3ca8-497d-b775-38a3d40791a1", "id": 3} v 0) v1 -- 0x7fb31006bd40 con 0x7fb31006ee60 2024-01-16T14:54:29.234 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.251+0000 7fb3067fc700 1 -- 172.21.15.93:0/2600813840 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b35b6724-3ca8-497d-b775-38a3d40791a1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb300017070 con 0x7fb31006ee60 2024-01-16T14:54:29.234 INFO:teuthology.orchestra.run.smithi093.stdout:3 2024-01-16T14:54:29.235 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 msgr2=0x7fb3100af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.235 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c4d0 tx=0x7fb30000c7e0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 shutdown_connections 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3100b5ab0 0x7fb3100c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb31006b620 0x7fb310136180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 --2- 172.21.15.93:0/2600813840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb31006ee60 0x7fb3100af060 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 >> 172.21.15.93:0/2600813840 conn(0x7fb3100b2740 msgr2=0x7fb3101464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 shutdown_connections 2024-01-16T14:54:29.236 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.252+0000 7fb3181cc700 1 -- 172.21.15.93:0/2600813840 wait complete. 2024-01-16T14:54:29.246 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 998722f9-d9d5-4927-a70e-079f6f407ce1 4 2024-01-16T14:54:29.297 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:29.314+0000 7f9c9b6b0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-16T14:54:29.368 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.385+0000 7f266a927700 1 Processor -- start 2024-01-16T14:54:29.369 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.385+0000 7f266a927700 1 -- start start 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f266a927700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f26640722c0 0x7f26640b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f266a927700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26641440f0 0x7f266406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f266a927700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2664143ec0 con 0x7f2664143ae0 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f266a927700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640b40a0 con 0x7f26641440f0 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f266a927700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26640b3ae0 con 0x7f26640722c0 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f26640722c0 0x7f26640b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2663fff700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f2664143ae0 0x7f2664071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40242/0 (socket says 172.21.15.93:40242) 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2663fff700 1 -- 172.21.15.93:0/3078314739 learned_addr learned my addr 172.21.15.93:0/3078314739 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2668ec4700 1 --2- 172.21.15.93:0/3078314739 >> v2:172.21.15.26:3300/0 conn(0x7f26641440f0 0x7f266406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 -- 172.21.15.93:0/3078314739 >> v2:172.21.15.26:3300/0 conn(0x7f26641440f0 msgr2=0x7f266406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 --2- 172.21.15.93:0/3078314739 >> v2:172.21.15.26:3300/0 conn(0x7f26641440f0 0x7f266406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 -- 172.21.15.93:0/3078314739 >> v1:172.21.15.93:6789/0 conn(0x7f2664143ae0 legacy=0x7f2664071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.371 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 -- 172.21.15.93:0/3078314739 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640b3dc0 con 0x7f26640722c0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2662ffd700 1 -- 172.21.15.93:0/3078314739 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942171119 0 0) 0x7f2664143ec0 con 0x7f2664143ae0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f26637fe700 1 --2- 172.21.15.93:0/3078314739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26640722c0 0x7f26640b51b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f26640d1b00 tx=0x7f2654009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0419e7b8713c42f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2662ffd700 1 -- 172.21.15.93:0/3078314739 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011830 con 0x7f26640722c0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2662ffd700 1 -- 172.21.15.93:0/3078314739 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654011990 con 0x7f26640722c0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.386+0000 7f2662ffd700 1 -- 172.21.15.93:0/3078314739 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011d20 con 0x7f26640722c0 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 -- 172.21.15.93:0/3078314739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26640722c0 msgr2=0x7f26640b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 --2- 172.21.15.93:0/3078314739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26640722c0 0x7f26640b51b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f26640d1b00 tx=0x7f2654009b20 comp rx=0 tx=0).stop 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 -- 172.21.15.93:0/3078314739 shutdown_connections 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 --2- 172.21.15.93:0/3078314739 >> v2:172.21.15.26:3300/0 conn(0x7f26641440f0 0x7f266406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 --2- 172.21.15.93:0/3078314739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26640722c0 0x7f26640b51b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 -- 172.21.15.93:0/3078314739 >> 172.21.15.93:0/3078314739 conn(0x7f26640b2740 msgr2=0x7f26640b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 -- 172.21.15.93:0/3078314739 shutdown_connections 2024-01-16T14:54:29.372 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.387+0000 7f266a927700 1 -- 172.21.15.93:0/3078314739 wait complete. 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.388+0000 7f266a927700 1 Processor -- start 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.388+0000 7f266a927700 1 -- start start 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26640722c0 0x7f266413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26641440f0 0x7f2664159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640c9d30 con 0x7f26641440f0 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640c9f40 con 0x7f2664143ae0 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f266a927700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640ca220 con 0x7f26640722c0 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f26637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f26637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44388/0 (socket says 172.21.15.93:44388) 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f26637fe700 1 -- 172.21.15.93:0/2389184547 learned_addr learned my addr 172.21.15.93:0/2389184547 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f2663fff700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26640722c0 0x7f266413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.375 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.389+0000 7f2668ec4700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26641440f0 0x7f2664159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 -- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26640722c0 msgr2=0x7f266413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26640722c0 0x7f266413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 -- 172.21.15.93:0/2389184547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26641440f0 msgr2=0x7f2664159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26641440f0 0x7f2664159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 -- 172.21.15.93:0/2389184547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540094a0 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f26637fe700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26540090b0 tx=0x7f2654004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540036a0 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265401f070 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654003a60 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640ca500 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640c7be0 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2654026480 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.390+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2654021070 con 0x7f2664143ae0 2024-01-16T14:54:29.376 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.391+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26641447e0 con 0x7f2664143ae0 2024-01-16T14:54:29.377 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.393+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f265402b020 con 0x7f2664143ae0 2024-01-16T14:54:29.377 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:29.394+0000 7f9c9b6b0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-16T14:54:29.460 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.477+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "998722f9-d9d5-4927-a70e-079f6f407ce1", "id": 4} v 0) v1 -- 0x7f2664075480 con 0x7f2664143ae0 2024-01-16T14:54:29.463 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.480+0000 7f2660ff9700 1 -- 172.21.15.93:0/2389184547 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "998722f9-d9d5-4927-a70e-079f6f407ce1", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2654015020 con 0x7f2664143ae0 2024-01-16T14:54:29.463 INFO:teuthology.orchestra.run.smithi093.stdout:4 2024-01-16T14:54:29.464 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 msgr2=0x7f266415a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26540090b0 tx=0x7f2654004750 comp rx=0 tx=0).stop 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 shutdown_connections 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26641440f0 0x7f2664159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26640722c0 0x7f266413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 --2- 172.21.15.93:0/2389184547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2664143ae0 0x7f266415a3a0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26540090b0 tx=0x7f2654004750 comp rx=0 tx=0).stop 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 >> 172.21.15.93:0/2389184547 conn(0x7f26640b2740 msgr2=0x7f266406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 shutdown_connections 2024-01-16T14:54:29.465 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.481+0000 7f266a927700 1 -- 172.21.15.93:0/2389184547 wait complete. 2024-01-16T14:54:29.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new f9b0daec-8bb4-4fe0-b123-1600aaab32cd 5 2024-01-16T14:54:29.600 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.617+0000 7f5a013e0700 1 Processor -- start 2024-01-16T14:54:29.601 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.617+0000 7f5a013e0700 1 -- start start 2024-01-16T14:54:29.602 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f5a013e0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 0x7f59fc0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.602 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f5a013e0700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f59fc0adda0 0x7f59fc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f5a013e0700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc0b0ec0 con 0x7f59fc15ae00 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f5a013e0700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc073610 con 0x7f59fc0adda0 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f5a013e0700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc15b1e0 con 0x7f59fc0b03b0 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f59faffd700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f59fc15ae00 0x7f59fc0ad690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40252/0 (socket says 172.21.15.93:40252) 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f59faffd700 1 -- 172.21.15.93:0/1157239789 learned_addr learned my addr 172.21.15.93:0/1157239789 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f59fa7fc700 1 --2- 172.21.15.93:0/1157239789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 0x7f59fc0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.618+0000 7f59fb7fe700 1 --2- 172.21.15.93:0/1157239789 >> v2:172.21.15.93:3300/0 conn(0x7f59fc0adda0 0x7f59fc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59f9ffb700 1 -- 172.21.15.93:0/1157239789 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751217312 0 0) 0x7f59fc0b0ec0 con 0x7f59fc15ae00 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59f9ffb700 1 -- 172.21.15.93:0/1157239789 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e40035b0 con 0x7f59fc15ae00 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59fa7fc700 1 -- 172.21.15.93:0/1157239789 >> v2:172.21.15.93:3300/0 conn(0x7f59fc0adda0 msgr2=0x7f59fc06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59fa7fc700 1 --2- 172.21.15.93:0/1157239789 >> v2:172.21.15.93:3300/0 conn(0x7f59fc0adda0 0x7f59fc06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59fa7fc700 1 -- 172.21.15.93:0/1157239789 >> v1:172.21.15.93:6789/0 conn(0x7f59fc15ae00 legacy=0x7f59fc0ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59fa7fc700 1 -- 172.21.15.93:0/1157239789 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc0b0900 con 0x7f59fc0b03b0 2024-01-16T14:54:29.603 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59fa7fc700 1 --2- 172.21.15.93:0/1157239789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 0x7f59fc0b5e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009870 tx=0x7f59f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d541fa886c0c1b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.604 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59f9ffb700 1 -- 172.21.15.93:0/1157239789 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011830 con 0x7f59fc0b03b0 2024-01-16T14:54:29.604 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59f9ffb700 1 -- 172.21.15.93:0/1157239789 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0011990 con 0x7f59fc0b03b0 2024-01-16T14:54:29.604 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.619+0000 7f59f9ffb700 1 -- 172.21.15.93:0/1157239789 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011d20 con 0x7f59fc0b03b0 2024-01-16T14:54:29.606 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 -- 172.21.15.93:0/1157239789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 msgr2=0x7f59fc0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.606 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 --2- 172.21.15.93:0/1157239789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 0x7f59fc0b5e40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009870 tx=0x7f59f0009b80 comp rx=0 tx=0).stop 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 -- 172.21.15.93:0/1157239789 shutdown_connections 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 --2- 172.21.15.93:0/1157239789 >> v2:172.21.15.93:3300/0 conn(0x7f59fc0adda0 0x7f59fc06b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 --2- 172.21.15.93:0/1157239789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc0b03b0 0x7f59fc0b5e40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 -- 172.21.15.93:0/1157239789 >> 172.21.15.93:0/1157239789 conn(0x7f59fc0b5070 msgr2=0x7f59fc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 -- 172.21.15.93:0/1157239789 shutdown_connections 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.620+0000 7f5a013e0700 1 -- 172.21.15.93:0/1157239789 wait complete. 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.621+0000 7f5a013e0700 1 Processor -- start 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.621+0000 7f5a013e0700 1 -- start start 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59fc0b03b0 0x7f59fc15cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc15ae00 0x7f59fc15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc148310 con 0x7f59fc0b03b0 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc14cd60 con 0x7f59fc0adda0 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f5a013e0700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc14a9f0 con 0x7f59fc15ae00 2024-01-16T14:54:29.607 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44404/0 (socket says 172.21.15.93:44404) 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 -- 172.21.15.93:0/4012899505 learned_addr learned my addr 172.21.15.93:0/4012899505 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59fb7fe700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc15ae00 0x7f59fc15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59fa7fc700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59fc0b03b0 0x7f59fc15cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 -- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc15ae00 msgr2=0x7f59fc15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc15ae00 0x7f59fc15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 -- 172.21.15.93:0/4012899505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59fc0b03b0 msgr2=0x7f59fc15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59fc0b03b0 0x7f59fc15cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.622+0000 7f59faffd700 1 -- 172.21.15.93:0/4012899505 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00094a0 con 0x7f59fc0adda0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f59faffd700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59ec001eb0 tx=0x7f59ec00c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec005d70 con 0x7f59fc0adda0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec0070d0 con 0x7f59fc0adda0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc14bf00 con 0x7f59fc0adda0 2024-01-16T14:54:29.608 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec007460 con 0x7f59fc0adda0 2024-01-16T14:54:29.609 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.623+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc14c4c0 con 0x7f59fc0adda0 2024-01-16T14:54:29.609 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.624+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59ec01c440 con 0x7f59fc0adda0 2024-01-16T14:54:29.609 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.624+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f59ec012980 con 0x7f59fc0adda0 2024-01-16T14:54:29.609 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.624+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc148480 con 0x7f59fc0adda0 2024-01-16T14:54:29.609 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.626+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f59ec017070 con 0x7f59fc0adda0 2024-01-16T14:54:29.630 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:29.647+0000 7f82bec3f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-16T14:54:29.693 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.710+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f9b0daec-8bb4-4fe0-b123-1600aaab32cd", "id": 5} v 0) v1 -- 0x7f59fc075480 con 0x7f59fc0adda0 2024-01-16T14:54:29.704 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.720+0000 7f59e37fe700 1 -- 172.21.15.93:0/4012899505 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9b0daec-8bb4-4fe0-b123-1600aaab32cd", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f59ec012b90 con 0x7f59fc0adda0 2024-01-16T14:54:29.704 INFO:teuthology.orchestra.run.smithi093.stdout:5 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 msgr2=0x7f59fc14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59ec001eb0 tx=0x7f59ec00c7b0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 shutdown_connections 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59fc15ae00 0x7f59fc15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59fc0adda0 0x7f59fc14a4b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 --2- 172.21.15.93:0/4012899505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59fc0b03b0 0x7f59fc15cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 >> 172.21.15.93:0/4012899505 conn(0x7f59fc0b5070 msgr2=0x7f59fc0ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.705 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 shutdown_connections 2024-01-16T14:54:29.706 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.721+0000 7f5a013e0700 1 -- 172.21.15.93:0/4012899505 wait complete. 2024-01-16T14:54:29.716 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 512d1b6a-54bf-4854-84f6-3623eb3b4047 6 2024-01-16T14:54:29.793 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:29.810+0000 7f9c9b6b0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-16T14:54:29.838 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.855+0000 7fa5c31f3700 1 Processor -- start 2024-01-16T14:54:29.839 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.855+0000 7fa5c31f3700 1 -- start start 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c31f3700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5bc15a570 0x7fa5bc0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c31f3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc0ae470 con 0x7fa5bc0b0490 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c31f3700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc0ad860 con 0x7fa5bc06bae0 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c31f3700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5bc0adb40 con 0x7fa5bc15a570 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5bbfff700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fa5bc06bae0 0x7fa5bc0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40260/0 (socket says 172.21.15.93:40260) 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5bbfff700 1 -- 172.21.15.93:0/133854027 learned_addr learned my addr 172.21.15.93:0/133854027 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c0f8f700 1 --2- 172.21.15.93:0/133854027 >> v2:172.21.15.26:3300/0 conn(0x7fa5bc15a570 0x7fa5bc0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5bb7fe700 1 -- 172.21.15.93:0/133854027 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200795291 0 0) 0x7fa5bc0ad860 con 0x7fa5bc06bae0 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5bb7fe700 1 -- 172.21.15.93:0/133854027 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac0035b0 con 0x7fa5bc06bae0 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c0f8f700 1 -- 172.21.15.93:0/133854027 >> v1:172.21.15.93:6789/0 conn(0x7fa5bc06bae0 legacy=0x7fa5bc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c0f8f700 1 -- 172.21.15.93:0/133854027 >> v1:172.21.15.26:6789/0 conn(0x7fa5bc0b0490 legacy=0x7fa5bc0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.841 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.856+0000 7fa5c0f8f700 1 -- 172.21.15.93:0/133854027 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0ade20 con 0x7fa5bc15a570 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c0f8f700 1 --2- 172.21.15.93:0/133854027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc0ad1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009840 tx=0x7fa5b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab5e7bdee2ceca0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5bb7fe700 1 -- 172.21.15.93:0/133854027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0011830 con 0x7fa5bc15a570 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5bb7fe700 1 -- 172.21.15.93:0/133854027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0011990 con 0x7fa5bc15a570 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5bb7fe700 1 -- 172.21.15.93:0/133854027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0011d20 con 0x7fa5bc15a570 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 -- 172.21.15.93:0/133854027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 msgr2=0x7fa5bc0ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/133854027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc0ad1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009840 tx=0x7fa5b0009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 -- 172.21.15.93:0/133854027 shutdown_connections 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/133854027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc0ad1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 -- 172.21.15.93:0/133854027 >> 172.21.15.93:0/133854027 conn(0x7fa5bc0717f0 msgr2=0x7fa5bc0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 -- 172.21.15.93:0/133854027 shutdown_connections 2024-01-16T14:54:29.842 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.857+0000 7fa5c31f3700 1 -- 172.21.15.93:0/133854027 wait complete. 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.858+0000 7fa5c31f3700 1 Processor -- start 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 -- start start 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1514f0 con 0x7fa5bc15a570 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1525b0 con 0x7fa5bc06bae0 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.859+0000 7fa5c31f3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc152890 con 0x7fa5bc0b0490 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c0f8f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c1790700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c0f8f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44418/0 (socket says 172.21.15.93:44418) 2024-01-16T14:54:29.845 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c0f8f700 1 -- 172.21.15.93:0/3526993767 learned_addr learned my addr 172.21.15.93:0/3526993767 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44632/0 (socket says 172.21.15.93:44632) 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 -- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 msgr2=0x7fa5bc14fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 -- 172.21.15.93:0/3526993767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 msgr2=0x7fa5bc154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 -- 172.21.15.93:0/3526993767 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00094a0 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c0f8f700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c1790700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5bbfff700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800c490 tx=0x7fa5a800c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8005d70 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a80070d0 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc152b70 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8007440 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc1511c0 con 0x7fa5bc0b0490 2024-01-16T14:54:29.846 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa5a801c440 con 0x7fa5bc0b0490 2024-01-16T14:54:29.847 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.860+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa5a80129f0 con 0x7fa5bc0b0490 2024-01-16T14:54:29.847 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.861+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc153090 con 0x7fa5bc0b0490 2024-01-16T14:54:29.847 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.863+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa5bc153090 con 0x7fa5bc0b0490 2024-01-16T14:54:29.861 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:29.877+0000 7f82bec3f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-16T14:54:29.931 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.947+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "512d1b6a-54bf-4854-84f6-3623eb3b4047", "id": 6} v 0) v1 -- 0x7fa5bc075480 con 0x7fa5bc0b0490 2024-01-16T14:54:29.936 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.953+0000 7fa5b97fa700 1 -- 172.21.15.93:0/3526993767 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "512d1b6a-54bf-4854-84f6-3623eb3b4047", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5a8017070 con 0x7fa5bc0b0490 2024-01-16T14:54:29.937 INFO:teuthology.orchestra.run.smithi093.stdout:6 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 msgr2=0x7fa5bc150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800c490 tx=0x7fa5a800c7a0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 shutdown_connections 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5bc15a570 0x7fa5bc154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5bc06bae0 0x7fa5bc14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 --2- 172.21.15.93:0/3526993767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5bc0b0490 0x7fa5bc150c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 >> 172.21.15.93:0/3526993767 conn(0x7fa5bc0717f0 msgr2=0x7fa5bc06e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:29.938 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 shutdown_connections 2024-01-16T14:54:29.939 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:29.954+0000 7fa5c31f3700 1 -- 172.21.15.93:0/3526993767 wait complete. 2024-01-16T14:54:29.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new e49da02e-f545-4805-8405-5425fac2faa3 7 2024-01-16T14:54:30.041 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:30.058+0000 7f9c9b6b0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-16T14:54:30.074 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.090+0000 7f457f885700 1 Processor -- start 2024-01-16T14:54:30.074 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 -- start start 2024-01-16T14:54:30.076 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f457815be40 0x7f45780af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.076 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f457806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.076 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45780af940 con 0x7f45780d0ec0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45780b1130 con 0x7f457815be40 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.091+0000 7f457f885700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4578067420 con 0x7f457806b7c0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457ce20700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f45780d0ec0 0x7f45780b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:40276/0 (socket says 172.21.15.93:40276) 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f457806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457ce20700 1 -- 172.21.15.93:0/4226124918 learned_addr learned my addr 172.21.15.93:0/4226124918 (peer_addr_for_me v1:172.21.15.93:0/0) 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457d621700 1 --2- 172.21.15.93:0/4226124918 >> v2:172.21.15.26:3300/0 conn(0x7f457815be40 0x7f45780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f456ffff700 1 -- 172.21.15.93:0/4226124918 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527722699 0 0) 0x7f4574001110 con 0x7f45780d0ec0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f456ffff700 1 -- 172.21.15.93:0/4226124918 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45640035b0 con 0x7f45780d0ec0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 -- 172.21.15.93:0/4226124918 >> v2:172.21.15.26:3300/0 conn(0x7f457815be40 msgr2=0x7f45780af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 --2- 172.21.15.93:0/4226124918 >> v2:172.21.15.26:3300/0 conn(0x7f457815be40 0x7f45780af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 -- 172.21.15.93:0/4226124918 >> v1:172.21.15.93:6789/0 conn(0x7f45780d0ec0 legacy=0x7f45780b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 -- 172.21.15.93:0/4226124918 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780b0ab0 con 0x7f457806b7c0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f457de22700 1 --2- 172.21.15.93:0/4226124918 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f457806bc50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4570009d90 tx=0x7f457000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c443c6efea676989 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f456ffff700 1 -- 172.21.15.93:0/4226124918 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000e070 con 0x7f457806b7c0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f456ffff700 1 -- 172.21.15.93:0/4226124918 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457000bb20 con 0x7f457806b7c0 2024-01-16T14:54:30.077 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.092+0000 7f456ffff700 1 -- 172.21.15.93:0/4226124918 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570012430 con 0x7f457806b7c0 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 -- 172.21.15.93:0/4226124918 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 msgr2=0x7f457806bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 --2- 172.21.15.93:0/4226124918 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f457806bc50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4570009d90 tx=0x7f457000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 -- 172.21.15.93:0/4226124918 shutdown_connections 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 --2- 172.21.15.93:0/4226124918 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f457806bc50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 --2- 172.21.15.93:0/4226124918 >> v2:172.21.15.26:3300/0 conn(0x7f457815be40 0x7f45780af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 -- 172.21.15.93:0/4226124918 >> 172.21.15.93:0/4226124918 conn(0x7f45780717f0 msgr2=0x7f45780b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 -- 172.21.15.93:0/4226124918 shutdown_connections 2024-01-16T14:54:30.078 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.093+0000 7f457f885700 1 -- 172.21.15.93:0/4226124918 wait complete. 2024-01-16T14:54:30.080 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.094+0000 7f457f885700 1 Processor -- start 2024-01-16T14:54:30.080 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.094+0000 7f457f885700 1 -- start start 2024-01-16T14:54:30.080 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 0x7f45781f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f457815be40 0x7f45781f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457809b1f0 con 0x7f457815be40 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457809b4d0 con 0x7f45780d0ec0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457f885700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457809b7b0 con 0x7f457806b7c0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457ce20700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 0x7f45781f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:44658/0 (socket says 172.21.15.93:44658) 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 -- 172.21.15.93:0/2126416480 learned_addr learned my addr 172.21.15.93:0/2126416480 (peer_addr_for_me v2:172.21.15.93:0/0) 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457de22700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f457815be40 0x7f45781f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 -- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 msgr2=0x7f45781f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 0x7f45781f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 -- 172.21.15.93:0/2126416480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f457815be40 msgr2=0x7f45781f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f457815be40 0x7f45781f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457d621700 1 -- 172.21.15.93:0/2126416480 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570009a70 con 0x7f457806b7c0 2024-01-16T14:54:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.095+0000 7f457ce20700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 0x7f45781f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f457d621700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f456800e870 tx=0x7f456800eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568009590 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568013680 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457809ba90 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680139f0 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45780b2810 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4568005a20 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.096+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f45680066b0 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.097+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457809bc80 con 0x7f457806b7c0 2024-01-16T14:54:30.082 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.099+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f45780b2810 con 0x7f457806b7c0 2024-01-16T14:54:30.107 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:30.123+0000 7f82bec3f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-16T14:54:30.167 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.184+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e49da02e-f545-4805-8405-5425fac2faa3", "id": 7} v 0) v1 -- 0x7f4578075480 con 0x7f457806b7c0 2024-01-16T14:54:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.189+0000 7f456dffb700 1 -- 172.21.15.93:0/2126416480 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e49da02e-f545-4805-8405-5425fac2faa3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4568021060 con 0x7f457806b7c0 2024-01-16T14:54:30.172 INFO:teuthology.orchestra.run.smithi093.stdout:7 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 msgr2=0x7f45780b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f456800e870 tx=0x7f456800eb80 comp rx=0 tx=0).stop 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 shutdown_connections 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f457815be40 0x7f45781f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f457806b7c0 0x7f45780b20f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 --2- 172.21.15.93:0/2126416480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45780d0ec0 0x7f45781f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 >> 172.21.15.93:0/2126416480 conn(0x7f45780717f0 msgr2=0x7f45780cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 shutdown_connections 2024-01-16T14:54:30.174 INFO:teuthology.orchestra.run.smithi093.stderr:2024-01-16T14:54:30.190+0000 7f457f885700 1 -- 172.21.15.93:0/2126416480 wait complete. 2024-01-16T14:54:30.185 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-16T14:54:30.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-16T14:54:30.188 INFO:tasks.ceph.osd.0:Started 2024-01-16T14:54:30.188 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-16T14:54:30.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-16T14:54:30.189 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:30.206+0000 7f82bec3f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-16T14:54:30.190 INFO:tasks.ceph.osd.1:Started 2024-01-16T14:54:30.190 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-16T14:54:30.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-16T14:54:30.192 INFO:tasks.ceph.osd.2:Started 2024-01-16T14:54:30.192 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-16T14:54:30.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-16T14:54:30.195 INFO:tasks.ceph.osd.3:Started 2024-01-16T14:54:30.195 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-16T14:54:30.195 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-16T14:54:30.197 INFO:tasks.ceph.osd.4:Started 2024-01-16T14:54:30.197 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-16T14:54:30.197 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-16T14:54:30.199 INFO:tasks.ceph.osd.5:Started 2024-01-16T14:54:30.199 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-16T14:54:30.200 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-16T14:54:30.202 INFO:tasks.ceph.osd.6:Started 2024-01-16T14:54:30.202 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-16T14:54:30.202 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-16T14:54:30.206 INFO:tasks.ceph.osd.7:Started 2024-01-16T14:54:30.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:30.357 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:30.374+0000 7f9c9b6b0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-16T14:54:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.388+0000 7f2816bcf700 1 Processor -- start 2024-01-16T14:54:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.388+0000 7f2816bcf700 1 -- start start 2024-01-16T14:54:30.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f2816bcf700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f281006f260 0x7f28100d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f2816bcf700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100ceea0 con 0x7f281006eca0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f2816bcf700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100ccf20 con 0x7f28100cd3b0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f2816bcf700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28100cf0d0 con 0x7f281006f260 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f280ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f281006eca0 0x7f281006f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60746/0 (socket says 172.21.15.26:60746) 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f280ffff700 1 -- 172.21.15.26:0/3825810502 learned_addr learned my addr 172.21.15.26:0/3825810502 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f281496b700 1 --2- 172.21.15.26:0/3825810502 >> v2:172.21.15.93:3300/0 conn(0x7f281006f260 0x7f28100d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.389+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700546387 0 0) 0x7f28100ceea0 con 0x7f281006eca0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f80035b0 con 0x7f281006eca0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246526715 0 0) 0x7f28100ccf20 con 0x7f28100cd3b0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28100ceea0 con 0x7f28100cd3b0 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f281496b700 1 -- 172.21.15.26:0/3825810502 >> v1:172.21.15.93:6789/0 conn(0x7f28100cd3b0 legacy=0x7f28100cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f281496b700 1 -- 172.21.15.26:0/3825810502 >> v1:172.21.15.26:6789/0 conn(0x7f281006eca0 legacy=0x7f281006f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f281496b700 1 -- 172.21.15.26:0/3825810502 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28100cf3b0 con 0x7f281006f260 2024-01-16T14:54:30.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f281496b700 1 --2- 172.21.15.26:0/3825810502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f28100d2700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2800009840 tx=0x7f2800009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209c1990ce34fa04 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800011830 con 0x7f281006f260 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800011990 con 0x7f281006f260 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.390+0000 7f280f7fe700 1 -- 172.21.15.26:0/3825810502 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800011d20 con 0x7f281006f260 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 -- 172.21.15.26:0/3825810502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 msgr2=0x7f28100d2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 --2- 172.21.15.26:0/3825810502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f28100d2700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2800009840 tx=0x7f2800009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 -- 172.21.15.26:0/3825810502 shutdown_connections 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 --2- 172.21.15.26:0/3825810502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f28100d2700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 -- 172.21.15.26:0/3825810502 >> 172.21.15.26:0/3825810502 conn(0x7f28100d1be0 msgr2=0x7f28100d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 -- 172.21.15.26:0/3825810502 shutdown_connections 2024-01-16T14:54:30.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.391+0000 7f2816bcf700 1 -- 172.21.15.26:0/3825810502 wait complete. 2024-01-16T14:54:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.392+0000 7f2816bcf700 1 Processor -- start 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.392+0000 7f2816bcf700 1 -- start start 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f2810148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28100cd3b0 0x7f2810148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28101576c0 con 0x7f281006eca0 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281014a210 con 0x7f281006f260 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f2816bcf700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28101476f0 con 0x7f28100cd3b0 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f281496b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f281496b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54096/0 (socket says 172.21.15.26:54096) 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f281496b700 1 -- 172.21.15.26:0/1999665180 learned_addr learned my addr 172.21.15.26:0/1999665180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f280ffff700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f2810148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.393+0000 7f281516c700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28100cd3b0 0x7f2810148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 -- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28100cd3b0 msgr2=0x7f2810148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28100cd3b0 0x7f2810148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 -- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 msgr2=0x7f2810148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f2810148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 -- 172.21.15.26:0/1999665180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000094a0 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f281496b700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2800009b90 tx=0x7f2800004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280001a6e0 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280001a860 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28101479d0 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280001ac10 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.394+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2810147f90 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.395+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810075480 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.395+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2800022070 con 0x7f281006eca0 2024-01-16T14:54:30.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.395+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f280001ad70 con 0x7f281006eca0 2024-01-16T14:54:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.398+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2800029070 con 0x7f281006eca0 2024-01-16T14:54:30.449 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:30.466+0000 7f9c9b6b0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-16T14:54:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.473+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28100b7b60 con 0x7f281006eca0 2024-01-16T14:54:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.474+0000 7f280d7fa700 1 -- 172.21.15.26:0/1999665180 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2800015020 con 0x7f281006eca0 2024-01-16T14:54:30.457 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:30.458 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:30.186010+0000","last_up_change":"0.000000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 msgr2=0x7f2810149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2800009b90 tx=0x7f2800004750 comp rx=0 tx=0).stop 2024-01-16T14:54:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 shutdown_connections 2024-01-16T14:54:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28100cd3b0 0x7f2810148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f281006eca0 0x7f2810149cd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 --2- 172.21.15.26:0/1999665180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f281006f260 0x7f2810148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 >> 172.21.15.26:0/1999665180 conn(0x7f28100d1be0 msgr2=0x7f2810159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 shutdown_connections 2024-01-16T14:54:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:30.475+0000 7f2816bcf700 1 -- 172.21.15.26:0/1999665180 wait complete. 2024-01-16T14:54:30.473 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-16T14:54:30.473 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-16T14:54:30.802 INFO:tasks.ceph.osd.6.smithi093.stderr:2024-01-16T14:54:30.818+0000 7f60dc35c640 -1 Falling back to public interface 2024-01-16T14:54:30.809 INFO:tasks.ceph.osd.4.smithi093.stderr:2024-01-16T14:54:30.826+0000 7f213c54b640 -1 Falling back to public interface 2024-01-16T14:54:30.815 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-16T14:54:30.832+0000 7f051a3b0640 -1 Falling back to public interface 2024-01-16T14:54:30.828 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-16T14:54:30.844+0000 7feac1c89640 -1 Falling back to public interface 2024-01-16T14:54:30.831 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-16T14:54:30.848+0000 7fabfb950640 -1 Falling back to public interface 2024-01-16T14:54:30.832 INFO:tasks.ceph.osd.7.smithi093.stderr:2024-01-16T14:54:30.849+0000 7f022570c640 -1 Falling back to public interface 2024-01-16T14:54:30.834 INFO:tasks.ceph.osd.5.smithi093.stderr:2024-01-16T14:54:30.851+0000 7ff78be99640 -1 Falling back to public interface 2024-01-16T14:54:30.842 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-16T14:54:30.859+0000 7f7edc296640 -1 Falling back to public interface 2024-01-16T14:54:30.925 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:30.942+0000 7f82bec3f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-16T14:54:31.005 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:31.022+0000 7f82bec3f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-16T14:54:31.139 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:31.155+0000 7f82bec3f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-16T14:54:31.271 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:31.287+0000 7f9c9b6b0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-16T14:54:31.276 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:31.364 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:31.381+0000 7f9c9b6b0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-16T14:54:31.402 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.418+0000 7fad3b2cb700 1 Processor -- start 2024-01-16T14:54:31.402 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.419+0000 7fad3b2cb700 1 -- start start 2024-01-16T14:54:31.406 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad3b2cb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 0x7fad340b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:31.406 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad3b2cb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 0x7fad34071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad3b2cb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad34072760 con 0x7fad340722d0 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad3b2cb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340b51d0 con 0x7fad340b4d50 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad3b2cb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3406e550 con 0x7fad3415abb0 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39868700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad340722d0 0x7fad3415e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60748/0 (socket says 172.21.15.26:60748) 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39868700 1 -- 172.21.15.26:0/42277344 learned_addr learned my addr 172.21.15.26:0/42277344 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad38866700 1 --2- 172.21.15.26:0/42277344 >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 0x7fad34071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39067700 1 --2- 172.21.15.26:0/42277344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 0x7fad340b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad2bfff700 1 -- 172.21.15.26:0/42277344 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553627539 0 0) 0x7fad2c001490 con 0x7fad340722d0 2024-01-16T14:54:31.407 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad2bfff700 1 -- 172.21.15.26:0/42277344 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad200035b0 con 0x7fad340722d0 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39067700 1 -- 172.21.15.26:0/42277344 >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 msgr2=0x7fad34071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39067700 1 --2- 172.21.15.26:0/42277344 >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 0x7fad34071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39067700 1 -- 172.21.15.26:0/42277344 >> v1:172.21.15.26:6789/0 conn(0x7fad340722d0 legacy=0x7fad3415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad39067700 1 -- 172.21.15.26:0/42277344 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3406f3b0 con 0x7fad3415abb0 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.420+0000 7fad38866700 1 --2- 172.21.15.26:0/42277344 >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 0x7fad34071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad39067700 1 --2- 172.21.15.26:0/42277344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 0x7fad340b4810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad30009840 tx=0x7fad30009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa612bd219bd92d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad2bfff700 1 -- 172.21.15.26:0/42277344 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30011830 con 0x7fad3415abb0 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad2bfff700 1 -- 172.21.15.26:0/42277344 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30011990 con 0x7fad3415abb0 2024-01-16T14:54:31.408 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad2bfff700 1 -- 172.21.15.26:0/42277344 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30011d20 con 0x7fad3415abb0 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 -- 172.21.15.26:0/42277344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 msgr2=0x7fad340b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/42277344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 0x7fad340b4810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad30009840 tx=0x7fad30009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 -- 172.21.15.26:0/42277344 shutdown_connections 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/42277344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad3415abb0 0x7fad340b4810 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/42277344 >> v2:172.21.15.93:3300/0 conn(0x7fad340b4d50 0x7fad34071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.421+0000 7fad3b2cb700 1 -- 172.21.15.26:0/42277344 >> 172.21.15.26:0/42277344 conn(0x7fad340d0aa0 msgr2=0x7fad340d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.422+0000 7fad3b2cb700 1 -- 172.21.15.26:0/42277344 shutdown_connections 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.422+0000 7fad3b2cb700 1 -- 172.21.15.26:0/42277344 wait complete. 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.423+0000 7fad3b2cb700 1 Processor -- start 2024-01-16T14:54:31.409 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.423+0000 7fad3b2cb700 1 -- start start 2024-01-16T14:54:31.411 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 0x7fad3415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad340b4d50 0x7fad34159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415b4d0 con 0x7fad3415abb0 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3414b550 con 0x7fad340722d0 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad3b2cb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415ce80 con 0x7fad340b4d50 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad38866700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad340b4d50 0x7fad34159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39067700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 0x7fad3415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54110/0 (socket says 172.21.15.26:54110) 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 -- 172.21.15.26:0/1338869324 learned_addr learned my addr 172.21.15.26:0/1338869324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 -- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad340b4d50 msgr2=0x7fad34159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad340b4d50 0x7fad34159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 -- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 msgr2=0x7fad3415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:31.412 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 0x7fad3415af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39868700 1 -- 172.21.15.26:0/1338869324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240094a0 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.424+0000 7fad39067700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 0x7fad3415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad39868700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00cb70 tx=0x7fad2c00ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00aa60 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00abc0 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad300094a0 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c012400 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3415b9e0 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fad2c012560 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.425+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fad2c01ba80 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.426+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3406ea60 con 0x7fad3415abb0 2024-01-16T14:54:31.413 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.429+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fad3406ea60 con 0x7fad3415abb0 2024-01-16T14:54:31.487 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.504+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad3414b700 con 0x7fad3415abb0 2024-01-16T14:54:31.488 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.505+0000 7fad29ffb700 1 -- 172.21.15.26:0/1338869324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fad3414b700 con 0x7fad3415abb0 2024-01-16T14:54:31.488 INFO:teuthology.misc.health.smithi026.stdout: 2024-01-16T14:54:31.488 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:30.186010+0000","last_up_change":"0.000000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 msgr2=0x7fad340ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00cb70 tx=0x7fad2c00ce80 comp rx=0 tx=0).stop 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 shutdown_connections 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415abb0 0x7fad340ceef0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad340722d0 0x7fad3415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 --2- 172.21.15.26:0/1338869324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad340b4d50 0x7fad34159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 >> 172.21.15.26:0/1338869324 conn(0x7fad340d0aa0 msgr2=0x7fad340b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 shutdown_connections 2024-01-16T14:54:31.490 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:31.506+0000 7fad3b2cb700 1 -- 172.21.15.26:0/1338869324 wait complete. 2024-01-16T14:54:31.502 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-16T14:54:31.519 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:31.536+0000 7f9c9b6b0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-16T14:54:31.690 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:31.707+0000 7f82bec3f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-16T14:54:31.859 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:31.876+0000 7f82bec3f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-16T14:54:32.025 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:32.041+0000 7f82bec3f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-16T14:54:32.157 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:32.174+0000 7f9c9b6b0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-16T14:54:32.347 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:32.364+0000 7f9c9b6b0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-16T14:54:32.534 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:32.551+0000 7f9c9b6b0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-16T14:54:32.839 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:32.855+0000 7f82bec3f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-16T14:54:33.062 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:33.079+0000 7f82bec3f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:54:33.135 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:33.152+0000 7f82bec3f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-16T14:54:33.308 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:33.324+0000 7f82bec3f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-16T14:54:33.480 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:33.497+0000 7f9c9b6b0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-16T14:54:33.795 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:33.812+0000 7f9c9b6b0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-16T14:54:33.879 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:33.896+0000 7f9c9b6b0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-16T14:54:34.070 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-16T14:54:34.087+0000 7f9c9b6b0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-16T14:54:34.278 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-16T14:54:34.295+0000 7f051a3b0640 -1 osd.1 0 log_to_monitors true 2024-01-16T14:54:34.296 INFO:tasks.ceph.osd.6.smithi093.stderr:2024-01-16T14:54:34.313+0000 7f60dc35c640 -1 osd.6 0 log_to_monitors true 2024-01-16T14:54:34.301 INFO:tasks.ceph.osd.4.smithi093.stderr:2024-01-16T14:54:34.318+0000 7f213c54b640 -1 osd.4 0 log_to_monitors true 2024-01-16T14:54:34.323 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-16T14:54:34.340+0000 7feac1c89640 -1 osd.0 0 log_to_monitors true 2024-01-16T14:54:34.327 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-16T14:54:34.343+0000 7fabfb950640 -1 osd.3 0 log_to_monitors true 2024-01-16T14:54:34.341 INFO:tasks.ceph.osd.5.smithi093.stderr:2024-01-16T14:54:34.356+0000 7ff78be99640 -1 osd.5 0 log_to_monitors true 2024-01-16T14:54:34.341 INFO:tasks.ceph.osd.7.smithi093.stderr:2024-01-16T14:54:34.355+0000 7f022570c640 -1 osd.7 0 log_to_monitors true 2024-01-16T14:54:34.341 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:34.358+0000 7f8279ff7700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-16T14:54:34.343 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:54:34.360+0000 7f8279ff7700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-16T14:54:34.350 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-16T14:54:34.367+0000 7f7edc296640 -1 osd.2 0 log_to_monitors true 2024-01-16T14:54:36.738 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-16T14:54:36.755+0000 7f7ecfe4a700 -1 osd.2 0 waiting for initial osdmap 2024-01-16T14:54:36.756 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-16T14:54:36.773+0000 7f7ec822a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:36.802 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-16T14:54:36.819+0000 7feab583d700 -1 osd.0 0 waiting for initial osdmap 2024-01-16T14:54:36.812 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-16T14:54:36.829+0000 7feaadc1d700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:36.814 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-16T14:54:36.831+0000 7fabf0d07700 -1 osd.3 0 waiting for initial osdmap 2024-01-16T14:54:36.819 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-16T14:54:36.836+0000 7f050df64700 -1 osd.1 0 waiting for initial osdmap 2024-01-16T14:54:36.825 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-16T14:54:36.842+0000 7fabe78e4700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:36.829 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-16T14:54:36.846+0000 7f0506344700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:37.401 INFO:tasks.ceph.osd.6.smithi093.stderr:2024-01-16T14:54:37.418+0000 7f60cff10700 -1 osd.6 0 waiting for initial osdmap 2024-01-16T14:54:37.402 INFO:tasks.ceph.osd.5.smithi093.stderr:2024-01-16T14:54:37.419+0000 7ff77fa4d700 -1 osd.5 0 waiting for initial osdmap 2024-01-16T14:54:37.408 INFO:tasks.ceph.osd.4.smithi093.stderr:2024-01-16T14:54:37.425+0000 7f21300ff700 -1 osd.4 0 waiting for initial osdmap 2024-01-16T14:54:37.414 INFO:tasks.ceph.osd.7.smithi093.stderr:2024-01-16T14:54:37.431+0000 7f02192c0700 -1 osd.7 0 waiting for initial osdmap 2024-01-16T14:54:37.414 INFO:tasks.ceph.osd.5.smithi093.stderr:2024-01-16T14:54:37.431+0000 7ff777e2d700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:37.418 INFO:tasks.ceph.osd.4.smithi093.stderr:2024-01-16T14:54:37.435+0000 7f21284df700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:37.418 INFO:tasks.ceph.osd.6.smithi093.stderr:2024-01-16T14:54:37.435+0000 7f60c82f0700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:37.425 INFO:tasks.ceph.osd.7.smithi093.stderr:2024-01-16T14:54:37.442+0000 7f02116a0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-16T14:54:38.308 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:38.443 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.460+0000 7f167b1e6700 1 Processor -- start 2024-01-16T14:54:38.443 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.460+0000 7f167b1e6700 1 -- start start 2024-01-16T14:54:38.445 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f167b1e6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16741457a0 0x7f16740cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.445 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f167b1e6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f167406b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f167b1e6700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740d04b0 con 0x7f16740cd470 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f167b1e6700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167406b920 con 0x7f16741457a0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f167b1e6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740cf1b0 con 0x7f167406afd0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16741457a0 0x7f16740cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1679783700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f167406b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f16741457a0 0x7f16740cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40820/0 (socket says 172.21.15.26:40820) 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 -- 172.21.15.26:0/1301554748 learned_addr learned my addr 172.21.15.26:0/1301554748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1673fff700 1 --1- 172.21.15.26:0/1301554748 >> v1:172.21.15.93:6789/0 conn(0x7f16740cd470 0x7f1674160330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:55482/0 (socket says 172.21.15.26:55482) 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 -- 172.21.15.26:0/1301554748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 msgr2=0x7f167406b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 --2- 172.21.15.26:0/1301554748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f167406b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 -- 172.21.15.26:0/1301554748 >> v1:172.21.15.93:6789/0 conn(0x7f16740cd470 legacy=0x7f1674160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 -- 172.21.15.26:0/1301554748 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740cf490 con 0x7f16741457a0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f16737fe700 1 -- 172.21.15.26:0/1301554748 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723988285 0 0) 0x7f16740d04b0 con 0x7f16740cd470 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f1678f82700 1 --2- 172.21.15.26:0/1301554748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16741457a0 0x7f16740cff70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f1668009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc7e265b8a47565 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.461+0000 7f16737fe700 1 -- 172.21.15.26:0/1301554748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011830 con 0x7f16741457a0 2024-01-16T14:54:38.446 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f16737fe700 1 -- 172.21.15.26:0/1301554748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668011990 con 0x7f16741457a0 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f16737fe700 1 -- 172.21.15.26:0/1301554748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668011d20 con 0x7f16741457a0 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 -- 172.21.15.26:0/1301554748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16741457a0 msgr2=0x7f16740cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 --2- 172.21.15.26:0/1301554748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16741457a0 0x7f16740cff70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f16680098f0 tx=0x7f1668009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 -- 172.21.15.26:0/1301554748 shutdown_connections 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 --2- 172.21.15.26:0/1301554748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f167406b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 --2- 172.21.15.26:0/1301554748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16741457a0 0x7f16740cff70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 -- 172.21.15.26:0/1301554748 >> 172.21.15.26:0/1301554748 conn(0x7f16740d0aa0 msgr2=0x7f16740d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 -- 172.21.15.26:0/1301554748 shutdown_connections 2024-01-16T14:54:38.447 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.462+0000 7f167b1e6700 1 -- 172.21.15.26:0/1301554748 wait complete. 2024-01-16T14:54:38.449 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.464+0000 7f167b1e6700 1 Processor -- start 2024-01-16T14:54:38.449 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.464+0000 7f167b1e6700 1 -- start start 2024-01-16T14:54:38.449 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f16740c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16741457a0 0x7f167413b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167413bb30 con 0x7f16740cd470 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16740c5fd0 con 0x7f16741457a0 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740ad730 con 0x7f167406afd0 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40826/0 (socket says 172.21.15.26:40826) 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 -- 172.21.15.26:0/345663399 learned_addr learned my addr 172.21.15.26:0/345663399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1678f82700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f16740c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1679783700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16741457a0 0x7f167413b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 -- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 msgr2=0x7f16740c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f16740c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 -- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16741457a0 msgr2=0x7f167413b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16741457a0 0x7f167413b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.450 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 -- 172.21.15.26:0/345663399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009550 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f1673fff700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f166000c550 tx=0x7f166000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660004c80 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660004e00 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660007310 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16740ada10 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.465+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16741f5020 con 0x7f16740cd470 2024-01-16T14:54:38.451 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.466+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16740ae2e0 con 0x7f16740cd470 2024-01-16T14:54:38.452 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.467+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f1660007470 con 0x7f16740cd470 2024-01-16T14:54:38.452 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.467+0000 7f16717fa700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 0x7f165806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.452 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.467+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f16600891b0 con 0x7f16740cd470 2024-01-16T14:54:38.452 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.468+0000 7f1678f82700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 0x7f165806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.452 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.468+0000 7f1678f82700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 0x7f165806d780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1668009c40 tx=0x7f1668008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.456 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.473+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16740ae2e0 con 0x7f16740cd470 2024-01-16T14:54:38.670 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.686+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16740ae2e0 con 0x7f16740cd470 2024-01-16T14:54:38.670 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.687+0000 7f16717fa700 1 -- 172.21.15.26:0/345663399 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10848 (secure 0 0 0) 0x7f16740ae2e0 con 0x7f16740cd470 2024-01-16T14:54:38.670 INFO:teuthology.misc.health.smithi026.stdout: 2024-01-16T14:54:38.671 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":13,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:37.775916+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:38.673 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 msgr2=0x7f165806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.673 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 0x7f165806d780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1668009c40 tx=0x7f1668008030 comp rx=0 tx=0).stop 2024-01-16T14:54:38.673 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 msgr2=0x7f167413b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.673 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f166000c550 tx=0x7f166000c860 comp rx=0 tx=0).stop 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 shutdown_connections 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16741457a0 0x7f167413b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f167406afd0 0x7f16740c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f165806b2c0 0x7f165806d780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 --2- 172.21.15.26:0/345663399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16740cd470 0x7f167413b0b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 >> 172.21.15.26:0/345663399 conn(0x7f16740d0aa0 msgr2=0x7f16740b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 shutdown_connections 2024-01-16T14:54:38.674 INFO:teuthology.misc.health.smithi026.stderr:2024-01-16T14:54:38.690+0000 7f167b1e6700 1 -- 172.21.15.26:0/345663399 wait complete. 2024-01-16T14:54:38.686 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-16T14:54:38.686 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-16T14:54:38.686 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-16T14:54:38.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-16T14:54:38.688 INFO:tasks.ceph.mds.a:Started 2024-01-16T14:54:38.689 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-16T14:54:38.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-16T14:54:38.691 INFO:tasks.ceph.mds.c:Started 2024-01-16T14:54:38.691 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-16T14:54:38.691 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-16T14:54:38.693 INFO:tasks.ceph.mds.b:Started 2024-01-16T14:54:38.693 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-16T14:54:38.693 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-16T14:54:38.696 INFO:tasks.ceph.mds.d:Started 2024-01-16T14:54:38.696 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-16T14:54:38.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:38.782 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-16T14:54:38.782 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-16T14:54:38.784 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2024-01-16T14:54:38.784 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2024-01-16T14:54:38.785 INFO:tasks.ceph.mds.b.smithi093.stderr:2024-01-16T14:54:38.802+0000 7f90228d5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-01-16T14:54:38.786 INFO:tasks.ceph.mds.d.smithi093.stderr:2024-01-16T14:54:38.803+0000 7f2eaa24b700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-01-16T14:54:38.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.860+0000 7f5c7f6dc700 1 Processor -- start 2024-01-16T14:54:38.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.860+0000 7f5c7f6dc700 1 -- start start 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7f6dc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c780af980 0x7f5c78066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7f6dc700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780b10c0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7f6dc700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780af730 con 0x7f5c7815bf10 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7f6dc700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780b1c60 con 0x7f5c780af980 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c780af980 0x7f5c78066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7dc79700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5c7806b7c0 0x7f5c780cccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51800/0 (socket says 172.21.15.26:51800) 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c780af980 0x7f5c78066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40864/0 (socket says 172.21.15.26:40864) 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7dc79700 1 -- 172.21.15.26:0/1905476049 learned_addr learned my addr 172.21.15.26:0/1905476049 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 -- 172.21.15.26:0/1905476049 >> v1:172.21.15.93:6789/0 conn(0x7f5c7815bf10 legacy=0x7f5c780b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 -- 172.21.15.26:0/1905476049 >> v1:172.21.15.26:6789/0 conn(0x7f5c7806b7c0 legacy=0x7f5c780cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c6ffff700 1 -- 172.21.15.26:0/1905476049 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661542302 0 0) 0x7f5c780b10c0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 -- 172.21.15.26:0/1905476049 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780b5ee0 con 0x7f5c780af980 2024-01-16T14:54:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.861+0000 7f5c7cc77700 1 --2- 172.21.15.26:0/1905476049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c780af980 0x7f5c78066ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009840 tx=0x7f5c68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b579183f6c75c53 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c6ffff700 1 -- 172.21.15.26:0/1905476049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011830 con 0x7f5c780af980 2024-01-16T14:54:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c6ffff700 1 -- 172.21.15.26:0/1905476049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68011990 con 0x7f5c780af980 2024-01-16T14:54:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c6ffff700 1 -- 172.21.15.26:0/1905476049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011d20 con 0x7f5c780af980 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/1905476049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c780af980 msgr2=0x7f5c78066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/1905476049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c780af980 0x7f5c78066ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009840 tx=0x7f5c68009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/1905476049 shutdown_connections 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/1905476049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c780af980 0x7f5c78066ce0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/1905476049 >> 172.21.15.26:0/1905476049 conn(0x7f5c780b5070 msgr2=0x7f5c780b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/1905476049 shutdown_connections 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.862+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/1905476049 wait complete. 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 Processor -- start 2024-01-16T14:54:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 -- start start 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 0x7f5c781571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 0x7f5c78157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815ef10 con 0x7f5c7806b7c0 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815e300 con 0x7f5c780af980 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.864+0000 7f5c7f6dc700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815e5e0 con 0x7f5c7815bf10 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40868/0 (socket says 172.21.15.26:40868) 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 -- 172.21.15.26:0/775433617 learned_addr learned my addr 172.21.15.26:0/775433617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7cc77700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 0x7f5c781571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7dc79700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 0x7f5c78157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 -- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 msgr2=0x7f5c78157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 0x7f5c78157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 -- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 msgr2=0x7f5c781571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 0x7f5c781571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 -- 172.21.15.26:0/775433617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680094a0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7cc77700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 0x7f5c781571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7dc79700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 0x7f5c78157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7d478700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c450 tx=0x7f5c7400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740052d0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c74005430 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7815e8c0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.865+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740038c0 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.866+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c781f3590 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.867+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78075480 con 0x7f5c7806b7c0 2024-01-16T14:54:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.867+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f5c74003a20 con 0x7f5c7806b7c0 2024-01-16T14:54:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.867+0000 7f5c6dffb700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 0x7f5c5c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.867+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5c74088950 con 0x7f5c7806b7c0 2024-01-16T14:54:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.868+0000 7f5c7cc77700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 0x7f5c5c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.868+0000 7f5c7cc77700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 0x7f5c5c06d780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c6801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:38.872+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c74053d70 con 0x7f5c7806b7c0 2024-01-16T14:54:39.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.086+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c780742f0 con 0x7f5c7806b7c0 2024-01-16T14:54:39.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.087+0000 7f5c6dffb700 1 -- 172.21.15.26:0/775433617 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+11032 (secure 0 0 0) 0x7f5c74057390 con 0x7f5c7806b7c0 2024-01-16T14:54:39.070 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:39.070 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":14,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:38.777626+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 msgr2=0x7f5c5c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 0x7f5c5c06d780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c6801c040 comp rx=0 tx=0).stop 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 msgr2=0x7f5c781f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c450 tx=0x7f5c7400c810 comp rx=0 tx=0).stop 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 shutdown_connections 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c7815bf10 0x7f5c78157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c5c06b2c0 0x7f5c5c06d780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7806b7c0 0x7f5c781f3050 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 --2- 172.21.15.26:0/775433617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c780af980 0x7f5c781571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 >> 172.21.15.26:0/775433617 conn(0x7f5c780b5070 msgr2=0x7f5c781594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 shutdown_connections 2024-01-16T14:54:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.090+0000 7f5c7f6dc700 1 -- 172.21.15.26:0/775433617 wait complete. 2024-01-16T14:54:39.087 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-01-16T14:54:39.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:54:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.240+0000 7f2cb1bb7700 1 Processor -- start 2024-01-16T14:54:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.240+0000 7f2cb1bb7700 1 -- start start 2024-01-16T14:54:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cb1bb7700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 0x7f2cac0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cb1bb7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cb1bb7700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0d04b0 con 0x7f2cac1457a0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cb1bb7700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac06b920 con 0x7f2cac0cd470 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cb1bb7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0cf1b0 con 0x7f2cac06afd0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2caaffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 0x7f2cac0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:52282/0 (socket says 172.21.15.26:52282) 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2caaffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 0x7f2cac0cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:57866/0 (socket says 172.21.15.26:57866) 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cabfff700 1 -- 172.21.15.26:0/950061638 learned_addr learned my addr 172.21.15.26:0/950061638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.241+0000 7f2cab7fe700 1 --1- 172.21.15.26:0/950061638 >> v1:172.21.15.93:6789/0 conn(0x7f2cac1457a0 0x7f2cac160330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:55498/0 (socket says 172.21.15.26:55498) 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2cabfff700 1 -- 172.21.15.26:0/950061638 >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 msgr2=0x7f2cac0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2cabfff700 1 --2- 172.21.15.26:0/950061638 >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 0x7f2cac0cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2cabfff700 1 -- 172.21.15.26:0/950061638 >> v1:172.21.15.93:6789/0 conn(0x7f2cac1457a0 legacy=0x7f2cac160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2cabfff700 1 -- 172.21.15.26:0/950061638 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0cf490 con 0x7f2cac06afd0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2cabfff700 1 --2- 172.21.15.26:0/950061638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009e50 tx=0x7f2ca400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5705debf28afc1b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2caa7fc700 1 -- 172.21.15.26:0/950061638 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400e070 con 0x7f2cac06afd0 2024-01-16T14:54:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2caa7fc700 1 -- 172.21.15.26:0/950061638 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400b910 con 0x7f2cac06afd0 2024-01-16T14:54:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.242+0000 7f2caa7fc700 1 -- 172.21.15.26:0/950061638 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400bcc0 con 0x7f2cac06afd0 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/950061638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 msgr2=0x7f2cac06b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/950061638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4009e50 tx=0x7f2ca400b330 comp rx=0 tx=0).stop 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/950061638 shutdown_connections 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/950061638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac06afd0 0x7f2cac06b3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/950061638 >> v2:172.21.15.93:3300/0 conn(0x7f2cac0cd470 0x7f2cac0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/950061638 >> 172.21.15.26:0/950061638 conn(0x7f2cac0d0aa0 msgr2=0x7f2cac0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/950061638 shutdown_connections 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.243+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/950061638 wait complete. 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.244+0000 7f2cb1bb7700 1 Processor -- start 2024-01-16T14:54:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.244+0000 7f2cb1bb7700 1 -- start start 2024-01-16T14:54:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac0cd470 0x7f2cac13c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cac1457a0 0x7f2cac0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0ae5e0 con 0x7f2cac06afd0 2024-01-16T14:54:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac140990 con 0x7f2cac1457a0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cb1bb7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac155710 con 0x7f2cac0cd470 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cab7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40876/0 (socket says 172.21.15.26:40876) 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cab7fe700 1 -- 172.21.15.26:0/1396376901 learned_addr learned my addr 172.21.15.26:0/1396376901 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2cabfff700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cac1457a0 0x7f2cac0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.245+0000 7f2caaffd700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac0cd470 0x7f2cac13c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 -- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac0cd470 msgr2=0x7f2cac13c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac0cd470 0x7f2cac13c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 -- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cac1457a0 msgr2=0x7f2cac0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cac1457a0 0x7f2cac0ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 -- 172.21.15.26:0/1396376901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca4009b30 con 0x7f2cac06afd0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cab7fe700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00c520 tx=0x7f2c9c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:39.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c005cb0 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c005e10 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac140c30 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0072d0 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.246+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1411f0 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.247+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac075480 con 0x7f2cac06afd0 2024-01-16T14:54:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.247+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f2c9c007430 con 0x7f2cac06afd0 2024-01-16T14:54:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.248+0000 7f2c93fff700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 0x7f2c9806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.248+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f2c9c088780 con 0x7f2cac06afd0 2024-01-16T14:54:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.248+0000 7f2caaffd700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 0x7f2c9806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.248+0000 7f2caaffd700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 0x7f2c9806d780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cac06be40 tx=0x7f2ca0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.253+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c9c053ce0 con 0x7f2cac06afd0 2024-01-16T14:54:39.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.514+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cac0d0880 con 0x7f2cac06afd0 2024-01-16T14:54:39.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.514+0000 7f2c93fff700 1 -- 172.21.15.26:0/1396376901 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f2c9c017090 con 0x7f2cac06afd0 2024-01-16T14:54:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:39.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 msgr2=0x7f2c9806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 0x7f2c9806d780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cac06be40 tx=0x7f2ca0009210 comp rx=0 tx=0).stop 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 msgr2=0x7f2cac1f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00c520 tx=0x7f2c9c00c830 comp rx=0 tx=0).stop 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 shutdown_connections 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cac1457a0 0x7f2cac0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2c9806b2c0 0x7f2c9806d780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cac06afd0 0x7f2cac1f6b60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 --2- 172.21.15.26:0/1396376901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cac0cd470 0x7f2cac13c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 >> 172.21.15.26:0/1396376901 conn(0x7f2cac0d0aa0 msgr2=0x7f2cac071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 shutdown_connections 2024-01-16T14:54:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:39.517+0000 7f2cb1bb7700 1 -- 172.21.15.26:0/1396376901 wait complete. 2024-01-16T14:54:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-01-16T14:54:41.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:54:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.679+0000 7f826451d700 1 Processor -- start 2024-01-16T14:54:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.679+0000 7f826451d700 1 -- start start 2024-01-16T14:54:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f826451d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f825c15e460 0x7f825c0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f826451d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f825c0cf480 0x7f825c0d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f826451d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825c0b3370 con 0x7f825c0d29d0 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f826451d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0d2e40 con 0x7f825c15e460 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f826451d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f825c0cfc40 con 0x7f825c0cf480 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f82622b9700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f825c15e460 0x7f825c0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f82622b9700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f825c15e460 0x7f825c0d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40878/0 (socket says 172.21.15.26:40878) 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f82622b9700 1 -- 172.21.15.26:0/3490848114 learned_addr learned my addr 172.21.15.26:0/3490848114 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.680+0000 7f8261ab8700 1 --2- 172.21.15.26:0/3490848114 >> v2:172.21.15.93:3300/0 conn(0x7f825c0cf480 0x7f825c0d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82622b9700 1 -- 172.21.15.26:0/3490848114 >> v2:172.21.15.93:3300/0 conn(0x7f825c0cf480 msgr2=0x7f825c0d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82622b9700 1 --2- 172.21.15.26:0/3490848114 >> v2:172.21.15.93:3300/0 conn(0x7f825c0cf480 0x7f825c0d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82622b9700 1 -- 172.21.15.26:0/3490848114 >> v1:172.21.15.93:6789/0 conn(0x7f825c0d29d0 legacy=0x7f825c0d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82622b9700 1 -- 172.21.15.26:0/3490848114 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0b35c0 con 0x7f825c15e460 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82612b7700 1 -- 172.21.15.26:0/3490848114 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117685809 0 0) 0x7f825c0b3370 con 0x7f825c0d29d0 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82622b9700 1 --2- 172.21.15.26:0/3490848114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0d49d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f825c08e2f0 tx=0x7f824c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3685f58e50bea6 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82612b7700 1 -- 172.21.15.26:0/3490848114 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00e070 con 0x7f825c15e460 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82612b7700 1 -- 172.21.15.26:0/3490848114 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00b910 con 0x7f825c15e460 2024-01-16T14:54:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.681+0000 7f82612b7700 1 -- 172.21.15.26:0/3490848114 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00bcc0 con 0x7f825c15e460 2024-01-16T14:54:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 -- 172.21.15.26:0/3490848114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 msgr2=0x7f825c0d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 --2- 172.21.15.26:0/3490848114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0d49d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f825c08e2f0 tx=0x7f824c00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 -- 172.21.15.26:0/3490848114 shutdown_connections 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 --2- 172.21.15.26:0/3490848114 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0d49d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 --2- 172.21.15.26:0/3490848114 >> v2:172.21.15.93:3300/0 conn(0x7f825c0cf480 0x7f825c0d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 -- 172.21.15.26:0/3490848114 >> 172.21.15.26:0/3490848114 conn(0x7f825c0d3eb0 msgr2=0x7f825c0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 -- 172.21.15.26:0/3490848114 shutdown_connections 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.682+0000 7f826451d700 1 -- 172.21.15.26:0/3490848114 wait complete. 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.683+0000 7f826451d700 1 Processor -- start 2024-01-16T14:54:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.683+0000 7f826451d700 1 -- start start 2024-01-16T14:54:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f825c0cf480 0x7f825c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c0d29d0 0x7f825c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cc880 con 0x7f825c15e460 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cb990 con 0x7f825c0d29d0 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f826451d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c0cbc70 con 0x7f825c0cf480 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f8262aba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f8262aba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40892/0 (socket says 172.21.15.26:40892) 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.684+0000 7f8262aba700 1 -- 172.21.15.26:0/862608622 learned_addr learned my addr 172.21.15.26:0/862608622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8261ab8700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c0d29d0 0x7f825c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f82622b9700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f825c0cf480 0x7f825c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 -- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f825c0cf480 msgr2=0x7f825c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f825c0cf480 0x7f825c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 -- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c0d29d0 msgr2=0x7f825c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c0d29d0 0x7f825c0c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 -- 172.21.15.26:0/862608622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c009b30 con 0x7f825c15e460 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f8262aba700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f825400c4d0 tx=0x7f825400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540054e0 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8254003680 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0cbf50 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254003a30 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.685+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825c0cc510 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.686+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825c0cca30 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.687+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f8254003b90 con 0x7f825c15e460 2024-01-16T14:54:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.687+0000 7f825affd700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 0x7f824806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.687+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8254088ae0 con 0x7f825c15e460 2024-01-16T14:54:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.693+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f825c0cca30 con 0x7f825c15e460 2024-01-16T14:54:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.707+0000 7f82622b9700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 0x7f824806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.707+0000 7f82622b9700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 0x7f824806d780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f824c009f90 tx=0x7f824c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.963+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f825c0cf900 con 0x7f825c15e460 2024-01-16T14:54:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.964+0000 7f825affd700 1 -- 172.21.15.26:0/862608622 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f8254017020 con 0x7f825c15e460 2024-01-16T14:54:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:41.947 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.966+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 msgr2=0x7f824806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.966+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 0x7f824806d780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f824c009f90 tx=0x7f824c009000 comp rx=0 tx=0).stop 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.966+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 msgr2=0x7f825c0c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.966+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f825400c4d0 tx=0x7f825400c7e0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 shutdown_connections 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f824806b2c0 0x7f824806d780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f825c15e460 0x7f825c0c9f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f825c0cf480 0x7f825c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 --2- 172.21.15.26:0/862608622 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f825c0d29d0 0x7f825c0c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 >> 172.21.15.26:0/862608622 conn(0x7f825c0d3eb0 msgr2=0x7f825c06cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 shutdown_connections 2024-01-16T14:54:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:41.967+0000 7f826451d700 1 -- 172.21.15.26:0/862608622 wait complete. 2024-01-16T14:54:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-01-16T14:54:41.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.118+0000 7fb6b2158700 1 Processor -- start 2024-01-16T14:54:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.118+0000 7fb6b2158700 1 -- start start 2024-01-16T14:54:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6b2158700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb6ac15e330 0x7fb6ac06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6b2158700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6b2158700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0cfe60 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6b2158700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0cf620 con 0x7fb6ac15e330 2024-01-16T14:54:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6b2158700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0d06e0 con 0x7fb6ac0d0b70 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6aaffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6ab7fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb6ac15e330 0x7fb6ac06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6aaffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:52306/0 (socket says 172.21.15.26:52306) 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6ab7fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb6ac15e330 0x7fb6ac06ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:57910/0 (socket says 172.21.15.26:57910) 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6aaffd700 1 -- 172.21.15.26:0/1732480056 learned_addr learned my addr 172.21.15.26:0/1732480056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.119+0000 7fb6abfff700 1 --1- 172.21.15.26:0/1732480056 >> v1:172.21.15.93:6789/0 conn(0x7fb6ac0d0fd0 0x7fb6ac0cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:55516/0 (socket says 172.21.15.26:55516) 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6ab7fe700 1 -- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 msgr2=0x7fb6ac06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6ab7fe700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6ab7fe700 1 -- 172.21.15.26:0/1732480056 >> v1:172.21.15.93:6789/0 conn(0x7fb6ac0d0fd0 legacy=0x7fb6ac0cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6ab7fe700 1 -- 172.21.15.26:0/1732480056 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac15e710 con 0x7fb6ac15e330 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6aa7fc700 1 -- 172.21.15.26:0/1732480056 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707033359 0 0) 0x7fb6ac0cfe60 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6aaffd700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6ab7fe700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac15e330 0x7fb6ac06ef40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009840 tx=0x7fb69c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a62ac35fea6bc9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6aa7fc700 1 -- 172.21.15.26:0/1732480056 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c011830 con 0x7fb6ac15e330 2024-01-16T14:54:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6aa7fc700 1 -- 172.21.15.26:0/1732480056 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c011990 con 0x7fb6ac15e330 2024-01-16T14:54:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.120+0000 7fb6aa7fc700 1 -- 172.21.15.26:0/1732480056 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c011d20 con 0x7fb6ac15e330 2024-01-16T14:54:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 -- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac15e330 msgr2=0x7fb6ac06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac15e330 0x7fb6ac06ef40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009840 tx=0x7fb69c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 -- 172.21.15.26:0/1732480056 shutdown_connections 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac15e330 0x7fb6ac06ef40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 --2- 172.21.15.26:0/1732480056 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac06f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 -- 172.21.15.26:0/1732480056 >> 172.21.15.26:0/1732480056 conn(0x7fb6ac0b3800 msgr2=0x7fb6ac0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 -- 172.21.15.26:0/1732480056 shutdown_connections 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.121+0000 7fb6b2158700 1 -- 172.21.15.26:0/1732480056 wait complete. 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.122+0000 7fb6b2158700 1 Processor -- start 2024-01-16T14:54:42.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.122+0000 7fb6b2158700 1 -- start start 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac14e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 0x7fb6ac0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 0x7fb6ac0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0c9d50 con 0x7fb6ac15e330 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0ca030 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6b2158700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0ca310 con 0x7fb6ac0d0b70 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 0x7fb6ac0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6aaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 0x7fb6ac0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6ab7fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac14e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 0x7fb6ac0c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40904/0 (socket says 172.21.15.26:40904) 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.123+0000 7fb6abfff700 1 -- 172.21.15.26:0/3385012713 learned_addr learned my addr 172.21.15.26:0/3385012713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 -- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 msgr2=0x7fb6ac14e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac14e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 -- 172.21.15.26:0/3385012713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 msgr2=0x7fb6ac0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 0x7fb6ac0c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 -- 172.21.15.26:0/3385012713 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009550 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6aaffd700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 0x7fb6ac0c5b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9a0 tx=0x7fb6a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004030 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0004190 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0004520 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c0094a0 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.124+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac0ca8d0 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.125+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac075480 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.125+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7fb6a00036a0 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.126+0000 7fb693fff700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 0x7fb69806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.126+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb6a0089520 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.126+0000 7fb6ab7fe700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 0x7fb69806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.126+0000 7fb6ab7fe700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 0x7fb69806d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009fd0 tx=0x7fb69c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.131+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6a00547c0 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.346+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6ac0b7c50 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.347+0000 7fb693fff700 1 -- 172.21.15.26:0/3385012713 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12790 (secure 0 0 0) 0x7fb6a0057de0 con 0x7fb6ac0d0fd0 2024-01-16T14:54:42.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:42.331 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:41.821083+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 msgr2=0x7fb69806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 0x7fb69806d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009fd0 tx=0x7fb69c01c040 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 msgr2=0x7fb6ac0c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 0x7fb6ac0c5b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9a0 tx=0x7fb6a000ecb0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 shutdown_connections 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb69806b2c0 0x7fb69806d780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac15e330 0x7fb6ac0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6ac0d0b70 0x7fb6ac14e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 --2- 172.21.15.26:0/3385012713 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6ac0d0fd0 0x7fb6ac0c5b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 >> 172.21.15.26:0/3385012713 conn(0x7fb6ac0b3800 msgr2=0x7fb6ac06c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 shutdown_connections 2024-01-16T14:54:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.350+0000 7fb6b2158700 1 -- 172.21.15.26:0/3385012713 wait complete. 2024-01-16T14:54:42.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:54:42.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:54:42.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.504+0000 7f041e5e8700 1 Processor -- start 2024-01-16T14:54:42.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 -- start start 2024-01-16T14:54:42.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f041806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f041806b510 0x7f041806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0418070170 con 0x7f0418145690 2024-01-16T14:54:42.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041806b9a0 con 0x7f041806b510 2024-01-16T14:54:42.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.505+0000 7f041e5e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04180cf640 con 0x7f04180cf040 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f041806b510 0x7f041806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0417fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0418145690 0x7f04181602b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51804/0 (socket says 172.21.15.26:51804) 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f041806b510 0x7f041806e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40906/0 (socket says 172.21.15.26:40906) 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0417fff700 1 -- 172.21.15.26:0/1023253944 learned_addr learned my addr 172.21.15.26:0/1023253944 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f04177fe700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f041806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0416ffd700 1 -- 172.21.15.26:0/1023253944 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064390027 0 0) 0x7f0418070170 con 0x7f0418145690 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0416ffd700 1 -- 172.21.15.26:0/1023253944 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04040035b0 con 0x7f0418145690 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 -- 172.21.15.26:0/1023253944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 msgr2=0x7f041806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f041806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 -- 172.21.15.26:0/1023253944 >> v1:172.21.15.26:6789/0 conn(0x7f0418145690 legacy=0x7f04181602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 -- 172.21.15.26:0/1023253944 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180cf920 con 0x7f041806b510 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f041cb85700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041806b510 0x7f041806e1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0410009d90 tx=0x7f041000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412c69e191064922 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0416ffd700 1 -- 172.21.15.26:0/1023253944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041000e070 con 0x7f041806b510 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0416ffd700 1 -- 172.21.15.26:0/1023253944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041000bb20 con 0x7f041806b510 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.506+0000 7f0416ffd700 1 -- 172.21.15.26:0/1023253944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410012430 con 0x7f041806b510 2024-01-16T14:54:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 -- 172.21.15.26:0/1023253944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041806b510 msgr2=0x7f041806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041806b510 0x7f041806e1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0410009d90 tx=0x7f041000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 -- 172.21.15.26:0/1023253944 shutdown_connections 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f041806b510 0x7f041806e1e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1023253944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f041806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 -- 172.21.15.26:0/1023253944 >> 172.21.15.26:0/1023253944 conn(0x7f04180b27b0 msgr2=0x7f04180b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 -- 172.21.15.26:0/1023253944 shutdown_connections 2024-01-16T14:54:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.507+0000 7f041e5e8700 1 -- 172.21.15.26:0/1023253944 wait complete. 2024-01-16T14:54:42.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.508+0000 7f041e5e8700 1 Processor -- start 2024-01-16T14:54:42.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 -- start start 2024-01-16T14:54:42.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 0x7f041813ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f0418159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041815c370 con 0x7f0418145690 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041813b3b0 con 0x7f041806b510 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.509+0000 7f041e5e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041815a2b0 con 0x7f04180cf040 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40916/0 (socket says 172.21.15.26:40916) 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 -- 172.21.15.26:0/1523379069 learned_addr learned my addr 172.21.15.26:0/1523379069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f0417fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 0x7f041813ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f04177fe700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f0418159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 -- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 msgr2=0x7f0418159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f0418159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 -- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 msgr2=0x7f041813ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 0x7f041813ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 -- 172.21.15.26:0/1523379069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0410009a70 con 0x7f0418145690 2024-01-16T14:54:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f0417fff700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 0x7f041813ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041cb85700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f04180b6090 tx=0x7f0410004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100129f0 con 0x7f0418145690 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410020070 con 0x7f0418145690 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041815d660 con 0x7f0418145690 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410012db0 con 0x7f0418145690 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.510+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041815c8a0 con 0x7f0418145690 2024-01-16T14:54:42.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.511+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0418141950 con 0x7f0418145690 2024-01-16T14:54:42.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.512+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f0410004040 con 0x7f0418145690 2024-01-16T14:54:42.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.512+0000 7f0414ff9700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 0x7f03f806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.513+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0410022070 con 0x7f0418145690 2024-01-16T14:54:42.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.513+0000 7f0417fff700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 0x7f03f806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.513+0000 7f0417fff700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 0x7f03f806d780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0408007fd0 tx=0x7f0408007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.518+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04100bde90 con 0x7f0418145690 2024-01-16T14:54:42.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.729+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0418075480 con 0x7f0418145690 2024-01-16T14:54:42.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.730+0000 7f0414ff9700 1 -- 172.21.15.26:0/1523379069 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f041002c0a0 con 0x7f0418145690 2024-01-16T14:54:42.713 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:54:42.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 msgr2=0x7f03f806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 0x7f03f806d780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0408007fd0 tx=0x7f0408007dc0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 msgr2=0x7f041815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f04180b6090 tx=0x7f0410004ad0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 shutdown_connections 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03f806b2c0 0x7f03f806d780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0418145690 0x7f041815aab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f041806b510 0x7f041813ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 --2- 172.21.15.26:0/1523379069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04180cf040 0x7f0418159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 >> 172.21.15.26:0/1523379069 conn(0x7f04180b27b0 msgr2=0x7f041806e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 shutdown_connections 2024-01-16T14:54:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.732+0000 7f041e5e8700 1 -- 172.21.15.26:0/1523379069 wait complete. 2024-01-16T14:54:42.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-16T14:54:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.883+0000 7f23adca3700 1 Processor -- start 2024-01-16T14:54:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.883+0000 7f23adca3700 1 -- start start 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23adca3700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f23a81478c0 0x7f23a80b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23adca3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23adca3700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a806dbe0 con 0x7f23a8161a90 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23adca3700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a8147ca0 con 0x7f23a81478c0 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23adca3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a81622e0 con 0x7f23a806b6e0 2024-01-16T14:54:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23a7fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23a7fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:52340/0 (socket says 172.21.15.26:52340) 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.884+0000 7f23a7fff700 1 -- 172.21.15.26:0/3541544584 learned_addr learned my addr 172.21.15.26:0/3541544584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a77fe700 1 --2- 172.21.15.26:0/3541544584 >> v2:172.21.15.93:3300/0 conn(0x7f23a81478c0 0x7f23a80b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a7fff700 1 -- 172.21.15.26:0/3541544584 >> v2:172.21.15.93:3300/0 conn(0x7f23a81478c0 msgr2=0x7f23a80b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a7fff700 1 --2- 172.21.15.26:0/3541544584 >> v2:172.21.15.93:3300/0 conn(0x7f23a81478c0 0x7f23a80b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a7fff700 1 -- 172.21.15.26:0/3541544584 >> v1:172.21.15.93:6789/0 conn(0x7f23a8161a90 legacy=0x7f23a806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a7fff700 1 -- 172.21.15.26:0/3541544584 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a81625c0 con 0x7f23a806b6e0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a7fff700 1 --2- 172.21.15.26:0/3541544584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009e50 tx=0x7f23a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18aa943a6b6317f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a67fc700 1 -- 172.21.15.26:0/3541544584 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000e070 con 0x7f23a806b6e0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a67fc700 1 -- 172.21.15.26:0/3541544584 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a000b910 con 0x7f23a806b6e0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.885+0000 7f23a67fc700 1 -- 172.21.15.26:0/3541544584 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000bcc0 con 0x7f23a806b6e0 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 -- 172.21.15.26:0/3541544584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 msgr2=0x7f23a806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 --2- 172.21.15.26:0/3541544584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009e50 tx=0x7f23a000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 -- 172.21.15.26:0/3541544584 shutdown_connections 2024-01-16T14:54:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 --2- 172.21.15.26:0/3541544584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a806b6e0 0x7f23a806e370 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 --2- 172.21.15.26:0/3541544584 >> v2:172.21.15.93:3300/0 conn(0x7f23a81478c0 0x7f23a80b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 -- 172.21.15.26:0/3541544584 >> 172.21.15.26:0/3541544584 conn(0x7f23a80d0aa0 msgr2=0x7f23a80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 -- 172.21.15.26:0/3541544584 shutdown_connections 2024-01-16T14:54:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.886+0000 7f23adca3700 1 -- 172.21.15.26:0/3541544584 wait complete. 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.887+0000 7f23adca3700 1 Processor -- start 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.887+0000 7f23adca3700 1 -- start start 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 0x7f23a815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a8161a90 0x7f23a80b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80af3d0 con 0x7f23a806b6e0 2024-01-16T14:54:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80b23c0 con 0x7f23a81478c0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23adca3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80ae7c0 con 0x7f23a8161a90 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40930/0 (socket says 172.21.15.26:40930) 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.888+0000 7f23a77fe700 1 -- 172.21.15.26:0/798240637 learned_addr learned my addr 172.21.15.26:0/798240637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a6ffd700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 0x7f23a815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a7fff700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a8161a90 0x7f23a80b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 -- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a8161a90 msgr2=0x7f23a80b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a8161a90 0x7f23a80b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 -- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 msgr2=0x7f23a815cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 0x7f23a815cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 -- 172.21.15.26:0/798240637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0009b30 con 0x7f23a806b6e0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a6ffd700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 0x7f23a815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23a77fe700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f239800e9b0 tx=0x7f239800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398009710 con 0x7f23a806b6e0 2024-01-16T14:54:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239800c850 con 0x7f23a806b6e0 2024-01-16T14:54:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a80aeaa0 con 0x7f23a806b6e0 2024-01-16T14:54:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239800cbc0 con 0x7f23a806b6e0 2024-01-16T14:54:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.889+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a80af060 con 0x7f23a806b6e0 2024-01-16T14:54:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.890+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a8075480 con 0x7f23a806b6e0 2024-01-16T14:54:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.891+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f239800cd20 con 0x7f23a806b6e0 2024-01-16T14:54:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.891+0000 7f239ffff700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 0x7f238806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.891+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2398089880 con 0x7f23a806b6e0 2024-01-16T14:54:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.891+0000 7f23a6ffd700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 0x7f238806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.892+0000 7f23a6ffd700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 0x7f238806d780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23a815dfb0 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:42.896+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2398054b20 con 0x7f23a806b6e0 2024-01-16T14:54:43.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.106+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23a80af6b0 con 0x7f23a806b6e0 2024-01-16T14:54:43.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.106+0000 7f239ffff700 1 -- 172.21.15.26:0/798240637 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2398021090 con 0x7f23a806b6e0 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:54:43.090 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:54:43.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.108+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 msgr2=0x7f238806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 0x7f238806d780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23a815dfb0 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-01-16T14:54:43.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 msgr2=0x7f23a80b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f239800e9b0 tx=0x7f239800ed70 comp rx=0 tx=0).stop 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 shutdown_connections 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a8161a90 0x7f23a80b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f238806b2c0 0x7f238806d780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a806b6e0 0x7f23a80b1e80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 --2- 172.21.15.26:0/798240637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a81478c0 0x7f23a815cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 >> 172.21.15.26:0/798240637 conn(0x7f23a80d0aa0 msgr2=0x7f23a81457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 shutdown_connections 2024-01-16T14:54:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.109+0000 7f23adca3700 1 -- 172.21.15.26:0/798240637 wait complete. 2024-01-16T14:54:43.106 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-16T14:54:43.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-16T14:54:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.263+0000 7fef7768f700 1 Processor -- start 2024-01-16T14:54:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.263+0000 7fef7768f700 1 -- start start 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.264+0000 7fef7768f700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fef700cf970 0x7fef7006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.264+0000 7fef7768f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700722d0 con 0x7fef70154b80 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.264+0000 7fef7768f700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef700cf650 con 0x7fef7006ec10 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.264+0000 7fef7768f700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef70154f60 con 0x7fef700cf970 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.264+0000 7fef7542b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef70154b80 0x7fef700ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51808/0 (socket says 172.21.15.26:51808) 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef7542b700 1 -- 172.21.15.26:0/441723673 learned_addr learned my addr 172.21.15.26:0/441723673 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef74c2a700 1 --2- 172.21.15.26:0/441723673 >> v2:172.21.15.93:3300/0 conn(0x7fef700cf970 0x7fef7006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710048965 0 0) 0x7fef700722d0 con 0x7fef70154b80 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef5c0046c0 con 0x7fef70154b80 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef74c2a700 1 -- 172.21.15.26:0/441723673 >> v1:172.21.15.93:6789/0 conn(0x7fef7006ec10 legacy=0x7fef7015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104790654 0 0) 0x7fef700cf650 con 0x7fef7006ec10 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef74c2a700 1 -- 172.21.15.26:0/441723673 >> v1:172.21.15.26:6789/0 conn(0x7fef70154b80 legacy=0x7fef700ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef74c2a700 1 -- 172.21.15.26:0/441723673 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70071750 con 0x7fef700cf970 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef74c2a700 1 --2- 172.21.15.26:0/441723673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef700cf970 0x7fef7006e6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef60009840 tx=0x7fef60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855aa325edb1ef51 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011830 con 0x7fef700cf970 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60011990 con 0x7fef700cf970 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.265+0000 7fef67fff700 1 -- 172.21.15.26:0/441723673 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60011d20 con 0x7fef700cf970 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 -- 172.21.15.26:0/441723673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef700cf970 msgr2=0x7fef7006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 --2- 172.21.15.26:0/441723673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef700cf970 0x7fef7006e6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef60009840 tx=0x7fef60009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 -- 172.21.15.26:0/441723673 shutdown_connections 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 --2- 172.21.15.26:0/441723673 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef700cf970 0x7fef7006e6d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 -- 172.21.15.26:0/441723673 >> 172.21.15.26:0/441723673 conn(0x7fef700b2040 msgr2=0x7fef700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 -- 172.21.15.26:0/441723673 shutdown_connections 2024-01-16T14:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.266+0000 7fef7768f700 1 -- 172.21.15.26:0/441723673 wait complete. 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.267+0000 7fef7768f700 1 Processor -- start 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.267+0000 7fef7768f700 1 -- start start 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef7006ec10 0x7fef701e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef700cf970 0x7fef7015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7015baf0 con 0x7fef70154b80 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701e6a50 con 0x7fef7006ec10 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7768f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7015aee0 con 0x7fef700cf970 2024-01-16T14:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef75c2c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef75c2c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40932/0 (socket says 172.21.15.26:40932) 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef75c2c700 1 -- 172.21.15.26:0/4076422039 learned_addr learned my addr 172.21.15.26:0/4076422039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef74c2a700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef700cf970 0x7fef7015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.268+0000 7fef7542b700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef7006ec10 0x7fef701e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 -- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef700cf970 msgr2=0x7fef7015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef700cf970 0x7fef7015c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 -- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef7006ec10 msgr2=0x7fef701e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef7006ec10 0x7fef701e6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 -- 172.21.15.26:0/4076422039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600094a0 con 0x7fef70154b80 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef75c2c700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c7a0 tx=0x7fef6800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68003e30 con 0x7fef70154b80 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68005070 con 0x7fef70154b80 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7015b1c0 con 0x7fef70154b80 2024-01-16T14:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68005420 con 0x7fef70154b80 2024-01-16T14:54:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.269+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7015b780 con 0x7fef70154b80 2024-01-16T14:54:43.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.270+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef701583d0 con 0x7fef70154b80 2024-01-16T14:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.271+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7fef68005580 con 0x7fef70154b80 2024-01-16T14:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.271+0000 7fef65ffb700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 0x7fef5406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.271+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef68088cd0 con 0x7fef70154b80 2024-01-16T14:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.271+0000 7fef7542b700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 0x7fef5406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.272+0000 7fef7542b700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 0x7fef5406d780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef6c007fd0 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:43.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.276+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef68053f60 con 0x7fef70154b80 2024-01-16T14:54:43.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.486+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fef70068670 con 0x7fef70154b80 2024-01-16T14:54:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.831+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fef68017070 con 0x7fef70154b80 2024-01-16T14:54:43.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.835+0000 7fef65ffb700 1 -- 172.21.15.26:0/4076422039 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fef68054330 con 0x7fef70154b80 2024-01-16T14:54:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 msgr2=0x7fef5406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 0x7fef5406d780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef6c007fd0 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 msgr2=0x7fef700b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c7a0 tx=0x7fef6800cab0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 shutdown_connections 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fef5406b2c0 0x7fef5406d780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef70154b80 0x7fef700b10f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef7006ec10 0x7fef701e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 --2- 172.21.15.26:0/4076422039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef700cf970 0x7fef7015c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.837+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 >> 172.21.15.26:0/4076422039 conn(0x7fef700b2040 msgr2=0x7fef700ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.838+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 shutdown_connections 2024-01-16T14:54:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:43.838+0000 7fef7768f700 1 -- 172.21.15.26:0/4076422039 wait complete. 2024-01-16T14:54:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-16T14:54:43.841 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-16T14:54:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.004+0000 7f3206acb700 1 Processor -- start 2024-01-16T14:54:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.004+0000 7f3206acb700 1 -- start start 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f320015afc0 0x7f32000730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f320006c5d0 0x7f32000b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32000b0d80 con 0x7f320015afc0 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320015f6b0 con 0x7f320006c5d0 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3206acb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320015eb40 con 0x7f32000b08f0 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3204867700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f320015afc0 0x7f32000730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3204867700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f320015afc0 0x7f32000730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40944/0 (socket says 172.21.15.26:40944) 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.005+0000 7f3204867700 1 -- 172.21.15.26:0/1385924411 learned_addr learned my addr 172.21.15.26:0/1385924411 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f31fffff700 1 --2- 172.21.15.26:0/1385924411 >> v2:172.21.15.93:3300/0 conn(0x7f320006c5d0 0x7f32000b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3205068700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 -- 172.21.15.26:0/1385924411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 msgr2=0x7f320006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 -- 172.21.15.26:0/1385924411 >> v2:172.21.15.93:3300/0 conn(0x7f320006c5d0 msgr2=0x7f32000b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 --2- 172.21.15.26:0/1385924411 >> v2:172.21.15.93:3300/0 conn(0x7f320006c5d0 0x7f32000b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 -- 172.21.15.26:0/1385924411 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000735f0 con 0x7f320015afc0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f3204867700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f32000730b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32000afe50 tx=0x7f31f000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff0e24c9f7257de server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f31ff7fe700 1 -- 172.21.15.26:0/1385924411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000dd30 con 0x7f320015afc0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f31ff7fe700 1 -- 172.21.15.26:0/1385924411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f000de90 con 0x7f320015afc0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.006+0000 7f31ff7fe700 1 -- 172.21.15.26:0/1385924411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0010690 con 0x7f320015afc0 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 -- 172.21.15.26:0/1385924411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 msgr2=0x7f32000730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f32000730b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32000afe50 tx=0x7f31f000be70 comp rx=0 tx=0).stop 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 -- 172.21.15.26:0/1385924411 shutdown_connections 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 --2- 172.21.15.26:0/1385924411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f32000730b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 --2- 172.21.15.26:0/1385924411 >> v2:172.21.15.93:3300/0 conn(0x7f320006c5d0 0x7f32000b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 -- 172.21.15.26:0/1385924411 >> 172.21.15.26:0/1385924411 conn(0x7f32000717f0 msgr2=0x7f32000b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 -- 172.21.15.26:0/1385924411 shutdown_connections 2024-01-16T14:54:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.007+0000 7f3206acb700 1 -- 172.21.15.26:0/1385924411 wait complete. 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.008+0000 7f3206acb700 1 Processor -- start 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.008+0000 7f3206acb700 1 -- start start 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f320006c5d0 0x7f3200156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320009d0b0 con 0x7f320015afc0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320014aad0 con 0x7f320006c5d0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3206acb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320009c4a0 con 0x7f32000b08f0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3205068700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3205068700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40958/0 (socket says 172.21.15.26:40958) 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3205068700 1 -- 172.21.15.26:0/2912012163 learned_addr learned my addr 172.21.15.26:0/2912012163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f3204867700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f320006c5d0 0x7f3200156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.009+0000 7f31fffff700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 -- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 msgr2=0x7f320014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320014a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 -- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f320006c5d0 msgr2=0x7f3200156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f320006c5d0 0x7f3200156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 -- 172.21.15.26:0/2912012163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f000b7f0 con 0x7f320015afc0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3205068700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007e40 tx=0x7f31f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800ed30 con 0x7f320015afc0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8014420 con 0x7f320015afc0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320009c780 con 0x7f320015afc0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f80147d0 con 0x7f320015afc0 2024-01-16T14:54:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.010+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320009cd40 con 0x7f320015afc0 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.011+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320009d260 con 0x7f320015afc0 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.012+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f31f8014930 con 0x7f320015afc0 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.012+0000 7f31fd7fa700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 0x7f31ec06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.012+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f31f8089fa0 con 0x7f320015afc0 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.012+0000 7f3204867700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 0x7f31ec06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.013+0000 7f3204867700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 0x7f31ec06d810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31f000bae0 tx=0x7f31f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.018+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31f80bce90 con 0x7f320015afc0 2024-01-16T14:54:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.228+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3200075480 con 0x7f320015afc0 2024-01-16T14:54:44.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.833+0000 7f31fd7fa700 1 -- 172.21.15.26:0/2912012163 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f320009d260 con 0x7f320015afc0 2024-01-16T14:54:44.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 msgr2=0x7f31ec06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 0x7f31ec06d810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31f000bae0 tx=0x7f31f0008040 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 msgr2=0x7f320009bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31f8007e40 tx=0x7f31f800e3c0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 shutdown_connections 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f31ec06b350 0x7f31ec06d810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015afc0 0x7f320009bc70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f320006c5d0 0x7f3200156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 --2- 172.21.15.26:0/2912012163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32000b08f0 0x7f320014a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.839+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 >> 172.21.15.26:0/2912012163 conn(0x7f32000717f0 msgr2=0x7f320015de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:44.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.840+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 shutdown_connections 2024-01-16T14:54:44.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:44.841+0000 7f3206acb700 1 -- 172.21.15.26:0/2912012163 wait complete. 2024-01-16T14:54:44.833 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-16T14:54:44.852 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-16T14:54:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.010+0000 7f98add02700 1 Processor -- start 2024-01-16T14:54:44.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.010+0000 7f98add02700 1 -- start start 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98add02700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98a81478c0 0x7f98a80b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98add02700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a806b6e0 0x7f98a806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98add02700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a806dbe0 con 0x7f98a8161a90 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98add02700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a8147ca0 con 0x7f98a81478c0 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98add02700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81622e0 con 0x7f98a806b6e0 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98a77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98a81478c0 0x7f98a80b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98a77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98a81478c0 0x7f98a80b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49368/0 (socket says 172.21.15.26:49368) 2024-01-16T14:54:44.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98a77fe700 1 -- 172.21.15.26:0/4290858162 learned_addr learned my addr 172.21.15.26:0/4290858162 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.011+0000 7f98a7fff700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a806b6e0 0x7f98a806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a77fe700 1 -- 172.21.15.26:0/4290858162 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a806b6e0 msgr2=0x7f98a806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a77fe700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a806b6e0 0x7f98a806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a77fe700 1 -- 172.21.15.26:0/4290858162 >> v1:172.21.15.93:6789/0 conn(0x7f98a8161a90 legacy=0x7f98a806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a77fe700 1 -- 172.21.15.26:0/4290858162 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a81625c0 con 0x7f98a81478c0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a77fe700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a81478c0 0x7f98a80b57b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98980098f0 tx=0x7f9898009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b371c27916b7d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a67fc700 1 -- 172.21.15.26:0/4290858162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898011830 con 0x7f98a81478c0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a67fc700 1 -- 172.21.15.26:0/4290858162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898011990 con 0x7f98a81478c0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.012+0000 7f98a67fc700 1 -- 172.21.15.26:0/4290858162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898011d20 con 0x7f98a81478c0 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 -- 172.21.15.26:0/4290858162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a81478c0 msgr2=0x7f98a80b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a81478c0 0x7f98a80b57b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f98980098f0 tx=0x7f9898009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 -- 172.21.15.26:0/4290858162 shutdown_connections 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a806b6e0 0x7f98a806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 --2- 172.21.15.26:0/4290858162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a81478c0 0x7f98a80b57b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:44.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 -- 172.21.15.26:0/4290858162 >> 172.21.15.26:0/4290858162 conn(0x7f98a80d0aa0 msgr2=0x7f98a80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 -- 172.21.15.26:0/4290858162 shutdown_connections 2024-01-16T14:54:44.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.013+0000 7f98add02700 1 -- 172.21.15.26:0/4290858162 wait complete. 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.014+0000 7f98add02700 1 Processor -- start 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.014+0000 7f98add02700 1 -- start start 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98a81478c0 0x7f98a815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a8161a90 0x7f98a815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a815dc30 con 0x7f98a806b6e0 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a815efb0 con 0x7f98a81478c0 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98add02700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a815f220 con 0x7f98a8161a90 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98a77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49376/0 (socket says 172.21.15.26:49376) 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98a77fe700 1 -- 172.21.15.26:0/1555598755 learned_addr learned my addr 172.21.15.26:0/1555598755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98a7fff700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a8161a90 0x7f98a815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:45.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.015+0000 7f98a6ffd700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98a81478c0 0x7f98a815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 -- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a8161a90 msgr2=0x7f98a815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a8161a90 0x7f98a815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 -- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98a81478c0 msgr2=0x7f98a815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98a81478c0 0x7f98a815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 -- 172.21.15.26:0/1555598755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898009550 con 0x7f98a806b6e0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98a77fe700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9898009c40 tx=0x7f9898004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898004530 con 0x7f98a806b6e0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989801f070 con 0x7f98a806b6e0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a815f500 con 0x7f98a806b6e0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98980038c0 con 0x7f98a806b6e0 2024-01-16T14:54:45.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.016+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a815fac0 con 0x7f98a806b6e0 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.017+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a806cf10 con 0x7f98a806b6e0 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.018+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9898021030 con 0x7f98a806b6e0 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.018+0000 7f988ffff700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 0x7f989406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.018+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f989808af80 con 0x7f98a806b6e0 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.018+0000 7f98a6ffd700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 0x7f989406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:45.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.019+0000 7f98a6ffd700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 0x7f989406d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f989c000f60 tx=0x7f989c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.023+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98980bde90 con 0x7f98a806b6e0 2024-01-16T14:54:45.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.279+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f98a8075480 con 0x7f98a806b6e0 2024-01-16T14:54:45.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.859+0000 7f988ffff700 1 -- 172.21.15.26:0/1555598755 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9898015020 con 0x7f98a806b6e0 2024-01-16T14:54:45.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.869+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 msgr2=0x7f989406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:45.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.869+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 0x7f989406d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f989c000f60 tx=0x7f989c009040 comp rx=0 tx=0).stop 2024-01-16T14:54:45.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.869+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 msgr2=0x7f98a815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:45.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.869+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9898009c40 tx=0x7f9898004750 comp rx=0 tx=0).stop 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.872+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 shutdown_connections 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.873+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98a8161a90 0x7f98a815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.873+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f989406b350 0x7f989406d810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.873+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a806b6e0 0x7f98a815d6f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.873+0000 7f98add02700 1 --2- 172.21.15.26:0/1555598755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98a81478c0 0x7f98a815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:45.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.873+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 >> 172.21.15.26:0/1555598755 conn(0x7f98a80d0aa0 msgr2=0x7f98a806afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:45.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.874+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 shutdown_connections 2024-01-16T14:54:45.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:45.875+0000 7f98add02700 1 -- 172.21.15.26:0/1555598755 wait complete. 2024-01-16T14:54:45.863 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-01-16T14:54:45.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.034+0000 7ffba54de700 1 Processor -- start 2024-01-16T14:54:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.034+0000 7ffba54de700 1 -- start start 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffba54de700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba015e460 0x7ffba00d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffba54de700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba00d2ba0 0x7ffba00b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffba54de700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba00b3370 con 0x7ffba00cf480 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffba54de700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba00cfc40 con 0x7ffba00d2ba0 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffba54de700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00b35c0 con 0x7ffba015e460 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffb9f7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba00d2ba0 0x7ffba00b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffb9f7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba00d2ba0 0x7ffba00b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49380/0 (socket says 172.21.15.26:49380) 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffb9f7fe700 1 -- 172.21.15.26:0/1017160540 learned_addr learned my addr 172.21.15.26:0/1017160540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffb9e7fc700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7ffba00cf480 0x7ffba00d2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49596/0 (socket says 172.21.15.26:49596) 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.035+0000 7ffb9effd700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba015e460 0x7ffba00d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9dffb700 1 -- 172.21.15.26:0/1017160540 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682048711 0 0) 0x7ffba00b3370 con 0x7ffba00cf480 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9dffb700 1 -- 172.21.15.26:0/1017160540 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb8c0035b0 con 0x7ffba00cf480 2024-01-16T14:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9f7fe700 1 -- 172.21.15.26:0/1017160540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba015e460 msgr2=0x7ffba00d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9f7fe700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba015e460 0x7ffba00d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9f7fe700 1 -- 172.21.15.26:0/1017160540 >> v1:172.21.15.93:6789/0 conn(0x7ffba00cf480 legacy=0x7ffba00d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9f7fe700 1 -- 172.21.15.26:0/1017160540 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba00b38a0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9f7fe700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba00b40a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c03341fdc9b284a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9dffb700 1 -- 172.21.15.26:0/1017160540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800e070 con 0x7ffba00d2ba0 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9dffb700 1 -- 172.21.15.26:0/1017160540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9800b910 con 0x7ffba00d2ba0 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.036+0000 7ffb9dffb700 1 -- 172.21.15.26:0/1017160540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800bc80 con 0x7ffba00d2ba0 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 -- 172.21.15.26:0/1017160540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 msgr2=0x7ffba00b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba00b40a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009e50 tx=0x7ffb9800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 -- 172.21.15.26:0/1017160540 shutdown_connections 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba00b40a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 --2- 172.21.15.26:0/1017160540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba015e460 0x7ffba00d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 -- 172.21.15.26:0/1017160540 >> 172.21.15.26:0/1017160540 conn(0x7ffba00d3eb0 msgr2=0x7ffba00d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 -- 172.21.15.26:0/1017160540 shutdown_connections 2024-01-16T14:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.037+0000 7ffba54de700 1 -- 172.21.15.26:0/1017160540 wait complete. 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.038+0000 7ffba54de700 1 Processor -- start 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.038+0000 7ffba54de700 1 -- start start 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba00cf480 0x7ffba00a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffba015e460 0x7ffba0155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00a14c0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00a1cb0 con 0x7ffba015e460 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffba54de700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba00cc720 con 0x7ffba00cf480 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49382/0 (socket says 172.21.15.26:49382) 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 -- 172.21.15.26:0/3659026148 learned_addr learned my addr 172.21.15.26:0/3659026148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9f7fe700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffba015e460 0x7ffba0155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9effd700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba00cf480 0x7ffba00a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 -- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba00cf480 msgr2=0x7ffba00a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba00cf480 0x7ffba00a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 -- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffba015e460 msgr2=0x7ffba0155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffba015e460 0x7ffba0155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.039+0000 7ffb9e7fc700 1 -- 172.21.15.26:0/3659026148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb98009b30 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffb9e7fc700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c500 tx=0x7ffb8800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88005cb0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb88005e10 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba009eff0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88007310 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.040+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba00a16f0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.041+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba00cc890 con 0x7ffba00d2ba0 2024-01-16T14:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.041+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ffb88007470 con 0x7ffba00d2ba0 2024-01-16T14:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.042+0000 7ffb977fe700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 0x7ffb8006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.042+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb88088ee0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.042+0000 7ffb9effd700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 0x7ffb8006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.043+0000 7ffb9effd700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 0x7ffb8006d810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009b40 tx=0x7ffb90009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.047+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb88053ea0 con 0x7ffba00d2ba0 2024-01-16T14:54:46.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.260+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffba0075480 con 0x7ffba00d2ba0 2024-01-16T14:54:46.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.261+0000 7ffb977fe700 1 -- 172.21.15.26:0/3659026148 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16458 (secure 0 0 0) 0x7ffb88017020 con 0x7ffba00d2ba0 2024-01-16T14:54:46.245 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:46.245 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:45.280554+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:46.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 msgr2=0x7ffb8006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 0x7ffb8006d810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009b40 tx=0x7ffb90009210 comp rx=0 tx=0).stop 2024-01-16T14:54:46.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 msgr2=0x7ffba009eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c500 tx=0x7ffb8800c810 comp rx=0 tx=0).stop 2024-01-16T14:54:46.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 shutdown_connections 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffba015e460 0x7ffba0155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffba00cf480 0x7ffba00a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ffb8006b350 0x7ffb8006d810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 --2- 172.21.15.26:0/3659026148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba00d2ba0 0x7ffba009eab0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 >> 172.21.15.26:0/3659026148 conn(0x7ffba00d3eb0 msgr2=0x7ffba00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 shutdown_connections 2024-01-16T14:54:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.264+0000 7ffba54de700 1 -- 172.21.15.26:0/3659026148 wait complete. 2024-01-16T14:54:46.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.415+0000 7f40d5e1d700 1 Processor -- start 2024-01-16T14:54:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.415+0000 7f40d5e1d700 1 -- start start 2024-01-16T14:54:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40d5e1d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f40d01478c0 0x7f40d00b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40d5e1d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40d5e1d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d006dbe0 con 0x7f40d006b510 2024-01-16T14:54:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40d5e1d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d006b9a0 con 0x7f40d01478c0 2024-01-16T14:54:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40d5e1d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0147ca0 con 0x7f40d0161a90 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40ceffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40cf7fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f40d01478c0 0x7f40d00b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40ceffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38220/0 (socket says 172.21.15.26:38220) 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40ceffd700 1 -- 172.21.15.26:0/1573379831 learned_addr learned my addr 172.21.15.26:0/1573379831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.416+0000 7f40cffff700 1 --1- 172.21.15.26:0/1573379831 >> v1:172.21.15.93:6789/0 conn(0x7f40d006b510 0x7f40d0071b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49606/0 (socket says 172.21.15.26:49606) 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ceffd700 1 -- 172.21.15.26:0/1573379831 >> v2:172.21.15.93:3300/0 conn(0x7f40d01478c0 msgr2=0x7f40d00b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ceffd700 1 --2- 172.21.15.26:0/1573379831 >> v2:172.21.15.93:3300/0 conn(0x7f40d01478c0 0x7f40d00b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ce7fc700 1 -- 172.21.15.26:0/1573379831 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048942302 0 0) 0x7f40d006dbe0 con 0x7f40d006b510 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ceffd700 1 -- 172.21.15.26:0/1573379831 >> v1:172.21.15.93:6789/0 conn(0x7f40d006b510 legacy=0x7f40d0071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ceffd700 1 -- 172.21.15.26:0/1573379831 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d01622e0 con 0x7f40d0161a90 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ceffd700 1 --2- 172.21.15.26:0/1573379831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098f0 tx=0x7f40c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58211778e89514f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ce7fc700 1 -- 172.21.15.26:0/1573379831 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011830 con 0x7f40d0161a90 2024-01-16T14:54:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ce7fc700 1 -- 172.21.15.26:0/1573379831 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4011990 con 0x7f40d0161a90 2024-01-16T14:54:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.417+0000 7f40ce7fc700 1 -- 172.21.15.26:0/1573379831 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4011d20 con 0x7f40d0161a90 2024-01-16T14:54:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1573379831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 msgr2=0x7f40d006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1573379831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f40c40098f0 tx=0x7f40c4009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1573379831 shutdown_connections 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1573379831 >> v2:172.21.15.93:3300/0 conn(0x7f40d01478c0 0x7f40d00b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1573379831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d006afd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1573379831 >> 172.21.15.26:0/1573379831 conn(0x7f40d00d0aa0 msgr2=0x7f40d00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1573379831 shutdown_connections 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.418+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1573379831 wait complete. 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.419+0000 7f40d5e1d700 1 Processor -- start 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.419+0000 7f40d5e1d700 1 -- start start 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40d01478c0 0x7f40d013bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d013c4b0 con 0x7f40d006b510 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0160eb0 con 0x7f40d01478c0 2024-01-16T14:54:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40d5e1d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00b0770 con 0x7f40d0161a90 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49392/0 (socket says 172.21.15.26:49392) 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40cf7fe700 1 -- 172.21.15.26:0/1822536458 learned_addr learned my addr 172.21.15.26:0/1822536458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40ceffd700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40d01478c0 0x7f40d013bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.420+0000 7f40cffff700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 -- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 msgr2=0x7f40d015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d015fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 -- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40d01478c0 msgr2=0x7f40d013bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40d01478c0 0x7f40d013bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 -- 172.21.15.26:0/1822536458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4009550 con 0x7f40d006b510 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40cf7fe700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40c000e9b0 tx=0x7f40c000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0009710 con 0x7f40d006b510 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000ee50 con 0x7f40d006b510 2024-01-16T14:54:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d00b0a50 con 0x7f40d006b510 2024-01-16T14:54:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c000ca50 con 0x7f40d006b510 2024-01-16T14:54:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.421+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d00b1010 con 0x7f40d006b510 2024-01-16T14:54:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.422+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d0075480 con 0x7f40d006b510 2024-01-16T14:54:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.423+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f40c000cbb0 con 0x7f40d006b510 2024-01-16T14:54:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.423+0000 7f40b7fff700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.423+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40c0089b20 con 0x7f40d006b510 2024-01-16T14:54:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.423+0000 7f40ceffd700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.424+0000 7f40ceffd700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 0x7f40bc06d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009c40 tx=0x7f40c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.428+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40c0054a20 con 0x7f40d006b510 2024-01-16T14:54:46.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.641+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d013c7d0 con 0x7f40d006b510 2024-01-16T14:54:46.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.642+0000 7f40b7fff700 1 -- 172.21.15.26:0/1822536458 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16458 (secure 0 0 0) 0x7f40c0021090 con 0x7f40d006b510 2024-01-16T14:54:46.626 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:46.626 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":20,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:45.280554+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 msgr2=0x7f40bc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 0x7f40bc06d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009c40 tx=0x7f40c4008030 comp rx=0 tx=0).stop 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 msgr2=0x7f40d013ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f40c000e9b0 tx=0x7f40c000ecc0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 shutdown_connections 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40d0161a90 0x7f40d015fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40d006b510 0x7f40d013ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.644+0000 7f40d5e1d700 1 --2- 172.21.15.26:0/1822536458 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40d01478c0 0x7f40d013bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.645+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 >> 172.21.15.26:0/1822536458 conn(0x7f40d00d0aa0 msgr2=0x7f40d01452c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.645+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 shutdown_connections 2024-01-16T14:54:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.645+0000 7f40d5e1d700 1 -- 172.21.15.26:0/1822536458 wait complete. 2024-01-16T14:54:46.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-16T14:54:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.797+0000 7f45ac5fc700 1 Processor -- start 2024-01-16T14:54:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.797+0000 7f45ac5fc700 1 -- start start 2024-01-16T14:54:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.798+0000 7f45ac5fc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a40ad1f0 0x7f45a40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.798+0000 7f45ac5fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45a40b02c0 0x7f45a40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.798+0000 7f45ac5fc700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a40ae470 con 0x7f45a415a570 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.798+0000 7f45ac5fc700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a40b0750 con 0x7f45a40b02c0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.798+0000 7f45ac5fc700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a40ad650 con 0x7f45a40ad1f0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aa398700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f45a415a570 0x7f45a406bae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58266/0 (socket says 172.21.15.26:58266) 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aa398700 1 -- 172.21.15.26:0/3142466110 learned_addr learned my addr 172.21.15.26:0/3142466110 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45a40b02c0 0x7f45a40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45a9b97700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a40ad1f0 0x7f45a40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 -- 172.21.15.26:0/3142466110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a40ad1f0 msgr2=0x7f45a40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a40ad1f0 0x7f45a40afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 -- 172.21.15.26:0/3142466110 >> v1:172.21.15.26:6789/0 conn(0x7f45a415a570 legacy=0x7f45a406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 -- 172.21.15.26:0/3142466110 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a40ad860 con 0x7f45a40b02c0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45aab99700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a40ae900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f459c009d90 tx=0x7f459c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdaa87d916e63 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45a9396700 1 -- 172.21.15.26:0/3142466110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c00bdd0 con 0x7f45a40b02c0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45a9396700 1 -- 172.21.15.26:0/3142466110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c011460 con 0x7f45a40b02c0 2024-01-16T14:54:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.799+0000 7f45a9396700 1 -- 172.21.15.26:0/3142466110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c011810 con 0x7f45a40b02c0 2024-01-16T14:54:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 -- 172.21.15.26:0/3142466110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 msgr2=0x7f45a40ae900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a40ae900 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f459c009d90 tx=0x7f459c00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 -- 172.21.15.26:0/3142466110 shutdown_connections 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a40ae900 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/3142466110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a40ad1f0 0x7f45a40afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 -- 172.21.15.26:0/3142466110 >> 172.21.15.26:0/3142466110 conn(0x7f45a40717f0 msgr2=0x7f45a40b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 -- 172.21.15.26:0/3142466110 shutdown_connections 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.800+0000 7f45ac5fc700 1 -- 172.21.15.26:0/3142466110 wait complete. 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.801+0000 7f45ac5fc700 1 Processor -- start 2024-01-16T14:54:46.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.801+0000 7f45ac5fc700 1 -- start start 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45a40ad1f0 0x7f45a4147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 0x7f45a409c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a409cf60 con 0x7f45a40b02c0 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a409d240 con 0x7f45a40ad1f0 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45ac5fc700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a409d520 con 0x7f45a415a570 2024-01-16T14:54:46.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45a9b97700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45a9b97700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49404/0 (socket says 172.21.15.26:49404) 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45a9b97700 1 -- 172.21.15.26:0/4198323943 learned_addr learned my addr 172.21.15.26:0/4198323943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45aab99700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 0x7f45a409c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.802+0000 7f45aa398700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45a40ad1f0 0x7f45a4147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 -- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 msgr2=0x7f45a409c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 0x7f45a409c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 -- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45a40ad1f0 msgr2=0x7f45a4147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45a40ad1f0 0x7f45a4147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 -- 172.21.15.26:0/4198323943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c009a70 con 0x7f45a40b02c0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45aab99700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 0x7f45a409c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45a9b97700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f459400d800 tx=0x7f459400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940125f0 con 0x7f45a40b02c0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594012750 con 0x7f45a40b02c0 2024-01-16T14:54:46.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a409d800 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594012b00 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.803+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a409a8b0 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.804+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a4075480 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.805+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4594012c60 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.805+0000 7f459affd700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 0x7f458806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.805+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4594089f40 con 0x7f45a40b02c0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.805+0000 7f45aa398700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 0x7f458806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:46.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.806+0000 7f45aa398700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 0x7f458806d810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005910 tx=0x7f45a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:46.810+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4594054db0 con 0x7f45a40b02c0 2024-01-16T14:54:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.074+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f45a409d0a0 con 0x7f45a40b02c0 2024-01-16T14:54:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.868+0000 7f459affd700 1 -- 172.21.15.26:0/4198323943 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f45a409d0a0 con 0x7f45a40b02c0 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 msgr2=0x7f458806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 0x7f458806d810 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005910 tx=0x7f45a0009040 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 msgr2=0x7f45a409a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f459400d800 tx=0x7f459400dbc0 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 shutdown_connections 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45a415a570 0x7f45a409c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45a40ad1f0 0x7f45a4147600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f458806b350 0x7f458806d810 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 --2- 172.21.15.26:0/4198323943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45a40b02c0 0x7f45a409a370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.871+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 >> 172.21.15.26:0/4198323943 conn(0x7f45a40717f0 msgr2=0x7f45a40b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.872+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 shutdown_connections 2024-01-16T14:54:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:47.872+0000 7f45ac5fc700 1 -- 172.21.15.26:0/4198323943 wait complete. 2024-01-16T14:54:47.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-16T14:54:47.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-16T14:54:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.031+0000 7f7553f41700 1 Processor -- start 2024-01-16T14:54:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.031+0000 7f7553f41700 1 -- start start 2024-01-16T14:54:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7553f41700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f754c15a570 0x7f754c0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7553f41700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f754c0b0490 0x7f754c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7553f41700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0ae470 con 0x7f754c06bae0 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7553f41700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0ad860 con 0x7f754c15a570 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7553f41700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0adb40 con 0x7f754c0b0490 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f754c15a570 0x7f754c0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f75514dc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f754c06bae0 0x7f754c0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58276/0 (socket says 172.21.15.26:58276) 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f75514dc700 1 -- 172.21.15.26:0/1804649162 learned_addr learned my addr 172.21.15.26:0/1804649162 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f75524de700 1 --2- 172.21.15.26:0/1804649162 >> v2:172.21.15.93:3300/0 conn(0x7f754c0b0490 0x7f754c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 -- 172.21.15.26:0/1804649162 >> v2:172.21.15.93:3300/0 conn(0x7f754c0b0490 msgr2=0x7f754c0ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 --2- 172.21.15.26:0/1804649162 >> v2:172.21.15.93:3300/0 conn(0x7f754c0b0490 0x7f754c0ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 -- 172.21.15.26:0/1804649162 >> v1:172.21.15.26:6789/0 conn(0x7f754c06bae0 legacy=0x7f754c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7550cdb700 1 -- 172.21.15.26:0/1804649162 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093332469 0 0) 0x7f754c0ae470 con 0x7f754c06bae0 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 -- 172.21.15.26:0/1804649162 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0ade20 con 0x7f754c15a570 2024-01-16T14:54:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.032+0000 7f7551cdd700 1 --2- 172.21.15.26:0/1804649162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c0ad1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f754c072090 tx=0x7f7548009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c19d86ab93862f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7550cdb700 1 -- 172.21.15.26:0/1804649162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011830 con 0x7f754c15a570 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7550cdb700 1 -- 172.21.15.26:0/1804649162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548011990 con 0x7f754c15a570 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7550cdb700 1 -- 172.21.15.26:0/1804649162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548011d20 con 0x7f754c15a570 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 -- 172.21.15.26:0/1804649162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 msgr2=0x7f754c0ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 --2- 172.21.15.26:0/1804649162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c0ad1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f754c072090 tx=0x7f7548009b20 comp rx=0 tx=0).stop 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 -- 172.21.15.26:0/1804649162 shutdown_connections 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 --2- 172.21.15.26:0/1804649162 >> v2:172.21.15.93:3300/0 conn(0x7f754c0b0490 0x7f754c0ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 --2- 172.21.15.26:0/1804649162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c0ad1f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 -- 172.21.15.26:0/1804649162 >> 172.21.15.26:0/1804649162 conn(0x7f754c0717f0 msgr2=0x7f754c0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 -- 172.21.15.26:0/1804649162 shutdown_connections 2024-01-16T14:54:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.033+0000 7f7553f41700 1 -- 172.21.15.26:0/1804649162 wait complete. 2024-01-16T14:54:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 Processor -- start 2024-01-16T14:54:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 -- start start 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f754c06bae0 0x7f754c147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f754c0b0490 0x7f754c1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c14d8b0 con 0x7f754c15a570 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c1f5ba0 con 0x7f754c06bae0 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f7553f41700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c1f6060 con 0x7f754c0b0490 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.035+0000 7f75524de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49422/0 (socket says 172.21.15.26:49422) 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 -- 172.21.15.26:0/360620237 learned_addr learned my addr 172.21.15.26:0/360620237 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f7551cdd700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f754c06bae0 0x7f754c147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75514dc700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f754c0b0490 0x7f754c1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 -- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f754c0b0490 msgr2=0x7f754c1f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f754c0b0490 0x7f754c1f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 -- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f754c06bae0 msgr2=0x7f754c147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f754c06bae0 0x7f754c147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 -- 172.21.15.26:0/360620237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480094a0 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f75524de700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7544007e40 tx=0x7f754400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754400ed30 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7544014420 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75440147d0 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c1485e0 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.036+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c1f5d80 con 0x7f754c15a570 2024-01-16T14:54:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.037+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c14d9f0 con 0x7f754c15a570 2024-01-16T14:54:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.038+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7544014930 con 0x7f754c15a570 2024-01-16T14:54:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.038+0000 7f753e7fc700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 0x7f753006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.038+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754408a1b0 con 0x7f754c15a570 2024-01-16T14:54:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.039+0000 7f7551cdd700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 0x7f753006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.039+0000 7f7551cdd700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 0x7f753006d810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75480090b0 tx=0x7f754801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.043+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f754c14d9f0 con 0x7f754c15a570 2024-01-16T14:54:48.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.301+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f754c14d9f0 con 0x7f754c15a570 2024-01-16T14:54:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.870+0000 7f753e7fc700 1 -- 172.21.15.26:0/360620237 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f754c14d9f0 con 0x7f754c15a570 2024-01-16T14:54:48.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 msgr2=0x7f753006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 0x7f753006d810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75480090b0 tx=0x7f754801c040 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 msgr2=0x7f754c1f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7544007e40 tx=0x7f754400e3c0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 shutdown_connections 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f753006b350 0x7f753006d810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c15a570 0x7f754c1f5660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f754c06bae0 0x7f754c147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 --2- 172.21.15.26:0/360620237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f754c0b0490 0x7f754c1f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 >> 172.21.15.26:0/360620237 conn(0x7f754c0717f0 msgr2=0x7f754c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 shutdown_connections 2024-01-16T14:54:48.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:48.874+0000 7f7553f41700 1 -- 172.21.15.26:0/360620237 wait complete. 2024-01-16T14:54:48.873 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-16T14:54:48.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:54:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.030+0000 7f8cda7eb700 1 Processor -- start 2024-01-16T14:54:49.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.030+0000 7f8cda7eb700 1 -- start start 2024-01-16T14:54:49.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cda7eb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8cd415bde0 0x7f8cd406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cda7eb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cda7eb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40cc670 con 0x7f8cd415c1c0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cda7eb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd40cffb0 con 0x7f8cd415bde0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cda7eb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40cf7a0 con 0x7f8cd40b4240 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd37fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8cd415c1c0 0x7f8cd40cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58292/0 (socket says 172.21.15.26:58292) 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd37fe700 1 -- 172.21.15.26:0/1533147342 learned_addr learned my addr 172.21.15.26:0/1533147342 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1533147342 >> v2:172.21.15.93:3300/0 conn(0x7f8cd415bde0 0x7f8cd406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd8d88700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd2ffd700 1 -- 172.21.15.26:0/1533147342 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815096181 0 0) 0x7f8cd40cc670 con 0x7f8cd415c1c0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.031+0000 7f8cd2ffd700 1 -- 172.21.15.26:0/1533147342 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc0035b0 con 0x7f8cd415c1c0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd3fff700 1 -- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 msgr2=0x7f8cd415fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd415fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd3fff700 1 -- 172.21.15.26:0/1533147342 >> v1:172.21.15.26:6789/0 conn(0x7f8cd415c1c0 legacy=0x7f8cd40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd3fff700 1 -- 172.21.15.26:0/1533147342 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40cf9f0 con 0x7f8cd415bde0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd8d88700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd415fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd406ece0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009840 tx=0x7f8cc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7516e318901db1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd2ffd700 1 -- 172.21.15.26:0/1533147342 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011830 con 0x7f8cd415bde0 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd2ffd700 1 -- 172.21.15.26:0/1533147342 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4011990 con 0x7f8cd415bde0 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.032+0000 7f8cd2ffd700 1 -- 172.21.15.26:0/1533147342 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011d20 con 0x7f8cd415bde0 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 msgr2=0x7f8cd406ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd406ece0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009840 tx=0x7f8cc4009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1533147342 shutdown_connections 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd415fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1533147342 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd406ece0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1533147342 >> 172.21.15.26:0/1533147342 conn(0x7f8cd40b3720 msgr2=0x7f8cd40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1533147342 shutdown_connections 2024-01-16T14:54:49.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.033+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1533147342 wait complete. 2024-01-16T14:54:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.034+0000 7f8cda7eb700 1 Processor -- start 2024-01-16T14:54:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.034+0000 7f8cda7eb700 1 -- start start 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd4151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c7f80 con 0x7f8cd4151430 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c76e0 con 0x7f8cd415bde0 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cda7eb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c79c0 con 0x7f8cd40b4240 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd8d88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd8d88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49438/0 (socket says 172.21.15.26:49438) 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd8d88700 1 -- 172.21.15.26:0/1884665422 learned_addr learned my addr 172.21.15.26:0/1884665422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd37fe700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd4151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd8d88700 1 -- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 msgr2=0x7f8cd4151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.035+0000 7f8cd8d88700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd4151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd8d88700 1 -- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 msgr2=0x7f8cd4150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd8d88700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd4150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd8d88700 1 -- 172.21.15.26:0/1884665422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc009a70 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd4151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd8d88700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00efc0 tx=0x7f8ccc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cb40 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00f040 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc003a20 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc40094a0 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.036+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd415c750 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.037+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd415c970 con 0x7f8cd4151430 2024-01-16T14:54:49.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.038+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8ccc014480 con 0x7f8cd4151430 2024-01-16T14:54:49.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.038+0000 7f8cd0ff9700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 0x7f8cc006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.038+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ccc08a640 con 0x7f8cd4151430 2024-01-16T14:54:49.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.038+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 0x7f8cc006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.039+0000 7f8cd3fff700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 0x7f8cc006d810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4000c00 tx=0x7f8cc4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.043+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ccc055430 con 0x7f8cd4151430 2024-01-16T14:54:49.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.299+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cd4075480 con 0x7f8cd4151430 2024-01-16T14:54:49.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.300+0000 7f8cd0ff9700 1 -- 172.21.15.26:0/1884665422 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f8ccc058a50 con 0x7f8cd4151430 2024-01-16T14:54:49.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:49.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 msgr2=0x7f8cc006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 0x7f8cc006d810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4000c00 tx=0x7f8cc4008030 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 msgr2=0x7f8cd40c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00efc0 tx=0x7f8ccc00c3a0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 shutdown_connections 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8cc006b350 0x7f8cc006d810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cd4151430 0x7f8cd40c69d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cd40b4240 0x7f8cd4151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 --2- 172.21.15.26:0/1884665422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cd415bde0 0x7f8cd4150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.302+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 >> 172.21.15.26:0/1884665422 conn(0x7f8cd40b3720 msgr2=0x7f8cd40afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.303+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 shutdown_connections 2024-01-16T14:54:49.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.303+0000 7f8cda7eb700 1 -- 172.21.15.26:0/1884665422 wait complete. 2024-01-16T14:54:49.287 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:54:49.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:49.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.453+0000 7f5adbb6a700 1 Processor -- start 2024-01-16T14:54:49.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.453+0000 7f5adbb6a700 1 -- start start 2024-01-16T14:54:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5adbb6a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ad40ad1f0 0x7f5ad40afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5adbb6a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5ad40b02c0 0x7f5ad40ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5adbb6a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad40b0710 con 0x7f5ad415a570 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5adbb6a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad40ad650 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5adbb6a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad40ae470 con 0x7f5ad40b02c0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5ad40ad1f0 0x7f5ad40afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9906700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ad415a570 0x7f5ad406bae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58302/0 (socket says 172.21.15.26:58302) 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9906700 1 -- 172.21.15.26:0/55143183 learned_addr learned my addr 172.21.15.26:0/55143183 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ada107700 1 --2- 172.21.15.26:0/55143183 >> v2:172.21.15.93:3300/0 conn(0x7f5ad40b02c0 0x7f5ad40ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 -- 172.21.15.26:0/55143183 >> v2:172.21.15.93:3300/0 conn(0x7f5ad40b02c0 msgr2=0x7f5ad40ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 --2- 172.21.15.26:0/55143183 >> v2:172.21.15.93:3300/0 conn(0x7f5ad40b02c0 0x7f5ad40ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 -- 172.21.15.26:0/55143183 >> v1:172.21.15.26:6789/0 conn(0x7f5ad415a570 legacy=0x7f5ad406bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 -- 172.21.15.26:0/55143183 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40ad860 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad8904700 1 -- 172.21.15.26:0/55143183 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455491282 0 0) 0x7f5ac8001110 con 0x7f5ad415a570 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.454+0000 7f5ad9105700 1 --2- 172.21.15.26:0/55143183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad40afd80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4072090 tx=0x7f5ad000acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ff83befc4a2dcc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5ad8904700 1 -- 172.21.15.26:0/55143183 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000cb70 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5ad8904700 1 -- 172.21.15.26:0/55143183 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad000ccd0 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5ad8904700 1 -- 172.21.15.26:0/55143183 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0018620 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 -- 172.21.15.26:0/55143183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad40ad1f0 msgr2=0x7f5ad40afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/55143183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad40afd80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4072090 tx=0x7f5ad000acc0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 -- 172.21.15.26:0/55143183 shutdown_connections 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/55143183 >> v2:172.21.15.93:3300/0 conn(0x7f5ad40b02c0 0x7f5ad40ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/55143183 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad40afd80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 -- 172.21.15.26:0/55143183 >> 172.21.15.26:0/55143183 conn(0x7f5ad40717f0 msgr2=0x7f5ad40b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 -- 172.21.15.26:0/55143183 shutdown_connections 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.455+0000 7f5adbb6a700 1 -- 172.21.15.26:0/55143183 wait complete. 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 Processor -- start 2024-01-16T14:54:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 -- start start 2024-01-16T14:54:49.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad41f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ad40b02c0 0x7f5ad4157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad41592e0 con 0x7f5ad415a570 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad414fb70 con 0x7f5ad40ad1f0 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5adbb6a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad41586d0 con 0x7f5ad40b02c0 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.457+0000 7f5ada107700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49452/0 (socket says 172.21.15.26:49452) 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 -- 172.21.15.26:0/1577768031 learned_addr learned my addr 172.21.15.26:0/1577768031 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ad9906700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad41f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ad9105700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ad40b02c0 0x7f5ad4157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 -- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ad40b02c0 msgr2=0x7f5ad4157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ad40b02c0 0x7f5ad4157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 -- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 msgr2=0x7f5ad41f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad41f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 -- 172.21.15.26:0/1577768031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad000a640 con 0x7f5ad415a570 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ad9906700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad41f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:49.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ada107700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00efc0 tx=0x7f5acc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00cd40 con 0x7f5ad415a570 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5acc00f040 con 0x7f5ad415a570 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad41589b0 con 0x7f5ad415a570 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc0146c0 con 0x7f5ad415a570 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.458+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad4158f70 con 0x7f5ad415a570 2024-01-16T14:54:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.459+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad4159450 con 0x7f5ad415a570 2024-01-16T14:54:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.460+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5acc014820 con 0x7f5ad415a570 2024-01-16T14:54:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.460+0000 7f5ac67fc700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 0x7f5ab806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.460+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5acc08a560 con 0x7f5ad415a570 2024-01-16T14:54:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.460+0000 7f5ad9906700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 0x7f5ab806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.461+0000 7f5ad9906700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 0x7f5ab806d810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.465+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5acc0553c0 con 0x7f5ad415a570 2024-01-16T14:54:49.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.678+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ad4075480 con 0x7f5ad415a570 2024-01-16T14:54:49.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.679+0000 7f5ac67fc700 1 -- 172.21.15.26:0/1577768031 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f5acc00cea0 con 0x7f5ad415a570 2024-01-16T14:54:49.663 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:49.663 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 msgr2=0x7f5ab806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 0x7f5ab806d810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8007fd0 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 msgr2=0x7f5ad4158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00efc0 tx=0x7f5acc00c3a0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 shutdown_connections 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ab806b350 0x7f5ab806d810 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ad415a570 0x7f5ad4158040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ad40ad1f0 0x7f5ad41f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 --2- 172.21.15.26:0/1577768031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ad40b02c0 0x7f5ad4157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 >> 172.21.15.26:0/1577768031 conn(0x7f5ad40717f0 msgr2=0x7f5ad406dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 shutdown_connections 2024-01-16T14:54:49.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.682+0000 7f5adbb6a700 1 -- 172.21.15.26:0/1577768031 wait complete. 2024-01-16T14:54:49.679 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-16T14:54:49.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:54:49.803 INFO:tasks.ceph.mds.b.smithi093.stderr:2024-01-16T14:54:49.820+0000 7f901c8c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T14:54:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.832+0000 7f6b13545700 1 Processor -- start 2024-01-16T14:54:49.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.832+0000 7f6b13545700 1 -- start start 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b13545700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b0c15abb0 0x7f6b0c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b13545700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c06e550 con 0x7f6b0c0724a0 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b13545700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c06f3b0 con 0x7f6b0c0b4d50 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b13545700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c0b51d0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b112e1700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b0c15abb0 0x7f6b0c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b11ae2700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b0c0724a0 0x7f6b0c0cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58314/0 (socket says 172.21.15.26:58314) 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b11ae2700 1 -- 172.21.15.26:0/311693221 learned_addr learned my addr 172.21.15.26:0/311693221 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b03fff700 1 -- 172.21.15.26:0/311693221 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20404418 0 0) 0x7f6b0c06e550 con 0x7f6b0c0724a0 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.833+0000 7f6b03fff700 1 -- 172.21.15.26:0/311693221 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af80035b0 con 0x7f6b0c0724a0 2024-01-16T14:54:49.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b112e1700 1 -- 172.21.15.26:0/311693221 >> v1:172.21.15.93:6789/0 conn(0x7f6b0c0b4d50 legacy=0x7f6b0c071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b112e1700 1 -- 172.21.15.26:0/311693221 >> v1:172.21.15.26:6789/0 conn(0x7f6b0c0724a0 legacy=0x7f6b0c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b112e1700 1 -- 172.21.15.26:0/311693221 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c06e7a0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b112e1700 1 --2- 172.21.15.26:0/311693221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c0b4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009840 tx=0x7f6b08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff607ba1312b3946 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b03fff700 1 -- 172.21.15.26:0/311693221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08004230 con 0x7f6b0c15abb0 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b03fff700 1 -- 172.21.15.26:0/311693221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08004390 con 0x7f6b0c15abb0 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b03fff700 1 -- 172.21.15.26:0/311693221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080036c0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b13545700 1 -- 172.21.15.26:0/311693221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 msgr2=0x7f6b0c0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b13545700 1 --2- 172.21.15.26:0/311693221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c0b4810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009840 tx=0x7f6b08009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b13545700 1 -- 172.21.15.26:0/311693221 shutdown_connections 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b13545700 1 --2- 172.21.15.26:0/311693221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c0b4810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.834+0000 7f6b13545700 1 -- 172.21.15.26:0/311693221 >> 172.21.15.26:0/311693221 conn(0x7f6b0c0d0aa0 msgr2=0x7f6b0c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.835+0000 7f6b13545700 1 -- 172.21.15.26:0/311693221 shutdown_connections 2024-01-16T14:54:49.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.835+0000 7f6b13545700 1 -- 172.21.15.26:0/311693221 wait complete. 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.836+0000 7f6b13545700 1 Processor -- start 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.836+0000 7f6b13545700 1 -- start start 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b0c0724a0 0x7f6b0c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b0c0b4d50 0x7f6b0c1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c154b50 con 0x7f6b0c15abb0 2024-01-16T14:54:49.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c157ed0 con 0x7f6b0c0724a0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b13545700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1581b0 con 0x7f6b0c0b4d50 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49468/0 (socket says 172.21.15.26:49468) 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 -- 172.21.15.26:0/3034172638 learned_addr learned my addr 172.21.15.26:0/3034172638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b112e1700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b0c0724a0 0x7f6b0c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b10ae0700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b0c0b4d50 0x7f6b0c1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 -- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b0c0b4d50 msgr2=0x7f6b0c1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b0c0b4d50 0x7f6b0c1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 -- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b0c0724a0 msgr2=0x7f6b0c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b0c0724a0 0x7f6b0c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 -- 172.21.15.26:0/3034172638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094a0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.837+0000 7f6b11ae2700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400cb40 tx=0x7f6b0400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.838+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400a9e0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.838+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0400ab40 con 0x7f6b0c15abb0 2024-01-16T14:54:49.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.838+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04012400 con 0x7f6b0c15abb0 2024-01-16T14:54:49.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.838+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c0a94f0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.838+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c0a9ab0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.839+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c06f8d0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.839+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6b0401b3b0 con 0x7f6b0c15abb0 2024-01-16T14:54:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.840+0000 7f6b01ffb700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 0x7f6af006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.840+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b04089770 con 0x7f6b0c15abb0 2024-01-16T14:54:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.840+0000 7f6b112e1700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 0x7f6af006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.841+0000 7f6b112e1700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 0x7f6af006d810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b080090b0 tx=0x7f6b08010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:49.845+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b04054690 con 0x7f6b0c15abb0 2024-01-16T14:54:50.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.055+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b0c072080 con 0x7f6af006b350 2024-01-16T14:54:50.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.063+0000 7f6b01ffb700 1 -- 172.21.15.26:0/3034172638 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264683 (secure 0 0 0) 0x7f6b0c072080 con 0x7f6af006b350 2024-01-16T14:54:50.050 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 msgr2=0x7f6af006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 0x7f6af006d810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b080090b0 tx=0x7f6b08010000 comp rx=0 tx=0).stop 2024-01-16T14:54:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 msgr2=0x7f6b0c154610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400cb40 tx=0x7f6b0400ce50 comp rx=0 tx=0).stop 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 shutdown_connections 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6af006b350 0x7f6af006d810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b0c15abb0 0x7f6b0c154610 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b0c0724a0 0x7f6b0c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 --2- 172.21.15.26:0/3034172638 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b0c0b4d50 0x7f6b0c1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 >> 172.21.15.26:0/3034172638 conn(0x7f6b0c0d0aa0 msgr2=0x7f6b0c0cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 shutdown_connections 2024-01-16T14:54:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.067+0000 7f6b13545700 1 -- 172.21.15.26:0/3034172638 wait complete. 2024-01-16T14:54:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:54:50.064 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-01-16T14:54:49.350728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2973696,"data_stored":1787728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.515941"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854111+0000","last_change":"2024-01-16T14:54:45.877716+0000","last_active":"2024-01-16T14:54:46.854111+0000","last_peered":"2024-01-16T14:54:46.854111+0000","last_clean":"2024-01-16T14:54:46.854111+0000","last_became_active":"2024-01-16T14:54:45.876462+0000","last_became_peered":"2024-01-16T14:54:45.876462+0000","last_unstale":"2024-01-16T14:54:46.854111+0000","last_undegraded":"2024-01-16T14:54:46.854111+0000","last_fullsized":"2024-01-16T14:54:46.854111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262923+0000","last_change":"2024-01-16T14:54:45.880623+0000","last_active":"2024-01-16T14:54:47.262923+0000","last_peered":"2024-01-16T14:54:47.262923+0000","last_clean":"2024-01-16T14:54:47.262923+0000","last_became_active":"2024-01-16T14:54:45.878584+0000","last_became_peered":"2024-01-16T14:54:45.878584+0000","last_unstale":"2024-01-16T14:54:47.262923+0000","last_undegraded":"2024-01-16T14:54:47.262923+0000","last_fullsized":"2024-01-16T14:54:47.262923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261939+0000","last_change":"2024-01-16T14:54:45.875065+0000","last_active":"2024-01-16T14:54:47.261939+0000","last_peered":"2024-01-16T14:54:47.261939+0000","last_clean":"2024-01-16T14:54:47.261939+0000","last_became_active":"2024-01-16T14:54:45.873809+0000","last_became_peered":"2024-01-16T14:54:45.873809+0000","last_unstale":"2024-01-16T14:54:47.261939+0000","last_undegraded":"2024-01-16T14:54:47.261939+0000","last_fullsized":"2024-01-16T14:54:47.261939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261991+0000","last_change":"2024-01-16T14:54:45.877233+0000","last_active":"2024-01-16T14:54:47.261991+0000","last_peered":"2024-01-16T14:54:47.261991+0000","last_clean":"2024-01-16T14:54:47.261991+0000","last_became_active":"2024-01-16T14:54:45.876940+0000","last_became_peered":"2024-01-16T14:54:45.876940+0000","last_unstale":"2024-01-16T14:54:47.261991+0000","last_undegraded":"2024-01-16T14:54:47.261991+0000","last_fullsized":"2024-01-16T14:54:47.261991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853429+0000","last_change":"2024-01-16T14:54:45.874709+0000","last_active":"2024-01-16T14:54:46.853429+0000","last_peered":"2024-01-16T14:54:46.853429+0000","last_clean":"2024-01-16T14:54:46.853429+0000","last_became_active":"2024-01-16T14:54:45.872900+0000","last_became_peered":"2024-01-16T14:54:45.872900+0000","last_unstale":"2024-01-16T14:54:46.853429+0000","last_undegraded":"2024-01-16T14:54:46.853429+0000","last_fullsized":"2024-01-16T14:54:46.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261215+0000","last_change":"2024-01-16T14:54:45.882475+0000","last_active":"2024-01-16T14:54:47.261215+0000","last_peered":"2024-01-16T14:54:47.261215+0000","last_clean":"2024-01-16T14:54:47.261215+0000","last_became_active":"2024-01-16T14:54:45.882150+0000","last_became_peered":"2024-01-16T14:54:45.882150+0000","last_unstale":"2024-01-16T14:54:47.261215+0000","last_undegraded":"2024-01-16T14:54:47.261215+0000","last_fullsized":"2024-01-16T14:54:47.261215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853589+0000","last_change":"2024-01-16T14:54:45.875750+0000","last_active":"2024-01-16T14:54:46.853589+0000","last_peered":"2024-01-16T14:54:46.853589+0000","last_clean":"2024-01-16T14:54:46.853589+0000","last_became_active":"2024-01-16T14:54:45.875474+0000","last_became_peered":"2024-01-16T14:54:45.875474+0000","last_unstale":"2024-01-16T14:54:46.853589+0000","last_undegraded":"2024-01-16T14:54:46.853589+0000","last_fullsized":"2024-01-16T14:54:46.853589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853586+0000","last_change":"2024-01-16T14:54:45.882976+0000","last_active":"2024-01-16T14:54:46.853586+0000","last_peered":"2024-01-16T14:54:46.853586+0000","last_clean":"2024-01-16T14:54:46.853586+0000","last_became_active":"2024-01-16T14:54:45.882628+0000","last_became_peered":"2024-01-16T14:54:45.882628+0000","last_unstale":"2024-01-16T14:54:46.853586+0000","last_undegraded":"2024-01-16T14:54:46.853586+0000","last_fullsized":"2024-01-16T14:54:46.853586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853730+0000","last_change":"2024-01-16T14:54:45.885967+0000","last_active":"2024-01-16T14:54:46.853730+0000","last_peered":"2024-01-16T14:54:46.853730+0000","last_clean":"2024-01-16T14:54:46.853730+0000","last_became_active":"2024-01-16T14:54:45.885700+0000","last_became_peered":"2024-01-16T14:54:45.885700+0000","last_unstale":"2024-01-16T14:54:46.853730+0000","last_undegraded":"2024-01-16T14:54:46.853730+0000","last_fullsized":"2024-01-16T14:54:46.853730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262320+0000","last_change":"2024-01-16T14:54:45.878748+0000","last_active":"2024-01-16T14:54:47.262320+0000","last_peered":"2024-01-16T14:54:47.262320+0000","last_clean":"2024-01-16T14:54:47.262320+0000","last_became_active":"2024-01-16T14:54:45.878446+0000","last_became_peered":"2024-01-16T14:54:45.878446+0000","last_unstale":"2024-01-16T14:54:47.262320+0000","last_undegraded":"2024-01-16T14:54:47.262320+0000","last_fullsized":"2024-01-16T14:54:47.262320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261357+0000","last_change":"2024-01-16T14:54:45.879232+0000","last_active":"2024-01-16T14:54:47.261357+0000","last_peered":"2024-01-16T14:54:47.261357+0000","last_clean":"2024-01-16T14:54:47.261357+0000","last_became_active":"2024-01-16T14:54:45.878875+0000","last_became_peered":"2024-01-16T14:54:45.878875+0000","last_unstale":"2024-01-16T14:54:47.261357+0000","last_undegraded":"2024-01-16T14:54:47.261357+0000","last_fullsized":"2024-01-16T14:54:47.261357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855341+0000","last_change":"2024-01-16T14:54:45.862373+0000","last_active":"2024-01-16T14:54:46.855341+0000","last_peered":"2024-01-16T14:54:46.855341+0000","last_clean":"2024-01-16T14:54:46.855341+0000","last_became_active":"2024-01-16T14:54:45.862110+0000","last_became_peered":"2024-01-16T14:54:45.862110+0000","last_unstale":"2024-01-16T14:54:46.855341+0000","last_undegraded":"2024-01-16T14:54:46.855341+0000","last_fullsized":"2024-01-16T14:54:46.855341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854063+0000","last_change":"2024-01-16T14:54:45.867064+0000","last_active":"2024-01-16T14:54:46.854063+0000","last_peered":"2024-01-16T14:54:46.854063+0000","last_clean":"2024-01-16T14:54:46.854063+0000","last_became_active":"2024-01-16T14:54:45.866257+0000","last_became_peered":"2024-01-16T14:54:45.866257+0000","last_unstale":"2024-01-16T14:54:46.854063+0000","last_undegraded":"2024-01-16T14:54:46.854063+0000","last_fullsized":"2024-01-16T14:54:46.854063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261852+0000","last_change":"2024-01-16T14:54:45.875851+0000","last_active":"2024-01-16T14:54:47.261852+0000","last_peered":"2024-01-16T14:54:47.261852+0000","last_clean":"2024-01-16T14:54:47.261852+0000","last_became_active":"2024-01-16T14:54:45.875202+0000","last_became_peered":"2024-01-16T14:54:45.875202+0000","last_unstale":"2024-01-16T14:54:47.261852+0000","last_undegraded":"2024-01-16T14:54:47.261852+0000","last_fullsized":"2024-01-16T14:54:47.261852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:35.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261833+0000","last_change":"2024-01-16T14:54:45.858130+0000","last_active":"2024-01-16T14:54:47.261833+0000","last_peered":"2024-01-16T14:54:47.261833+0000","last_clean":"2024-01-16T14:54:47.261833+0000","last_became_active":"2024-01-16T14:54:45.856812+0000","last_became_peered":"2024-01-16T14:54:45.856812+0000","last_unstale":"2024-01-16T14:54:47.261833+0000","last_undegraded":"2024-01-16T14:54:47.261833+0000","last_fullsized":"2024-01-16T14:54:47.261833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261530+0000","last_change":"2024-01-16T14:54:45.883516+0000","last_active":"2024-01-16T14:54:47.261530+0000","last_peered":"2024-01-16T14:54:47.261530+0000","last_clean":"2024-01-16T14:54:47.261530+0000","last_became_active":"2024-01-16T14:54:45.883259+0000","last_became_peered":"2024-01-16T14:54:45.883259+0000","last_unstale":"2024-01-16T14:54:47.261530+0000","last_undegraded":"2024-01-16T14:54:47.261530+0000","last_fullsized":"2024-01-16T14:54:47.261530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262193+0000","last_change":"2024-01-16T14:54:45.883796+0000","last_active":"2024-01-16T14:54:47.262193+0000","last_peered":"2024-01-16T14:54:47.262193+0000","last_clean":"2024-01-16T14:54:47.262193+0000","last_became_active":"2024-01-16T14:54:45.883558+0000","last_became_peered":"2024-01-16T14:54:45.883558+0000","last_unstale":"2024-01-16T14:54:47.262193+0000","last_undegraded":"2024-01-16T14:54:47.262193+0000","last_fullsized":"2024-01-16T14:54:47.262193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262894+0000","last_change":"2024-01-16T14:54:45.867763+0000","last_active":"2024-01-16T14:54:47.262894+0000","last_peered":"2024-01-16T14:54:47.262894+0000","last_clean":"2024-01-16T14:54:47.262894+0000","last_became_active":"2024-01-16T14:54:45.867441+0000","last_became_peered":"2024-01-16T14:54:45.867441+0000","last_unstale":"2024-01-16T14:54:47.262894+0000","last_undegraded":"2024-01-16T14:54:47.262894+0000","last_fullsized":"2024-01-16T14:54:47.262894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853971+0000","last_change":"2024-01-16T14:54:45.873151+0000","last_active":"2024-01-16T14:54:46.853971+0000","last_peered":"2024-01-16T14:54:46.853971+0000","last_clean":"2024-01-16T14:54:46.853971+0000","last_became_active":"2024-01-16T14:54:45.872841+0000","last_became_peered":"2024-01-16T14:54:45.872841+0000","last_unstale":"2024-01-16T14:54:46.853971+0000","last_undegraded":"2024-01-16T14:54:46.853971+0000","last_fullsized":"2024-01-16T14:54:46.853971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262224+0000","last_change":"2024-01-16T14:54:44.860003+0000","last_active":"2024-01-16T14:54:47.262224+0000","last_peered":"2024-01-16T14:54:47.262224+0000","last_clean":"2024-01-16T14:54:47.262224+0000","last_became_active":"2024-01-16T14:54:44.859774+0000","last_became_peered":"2024-01-16T14:54:44.859774+0000","last_unstale":"2024-01-16T14:54:47.262224+0000","last_undegraded":"2024-01-16T14:54:47.262224+0000","last_fullsized":"2024-01-16T14:54:47.262224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854240+0000","last_change":"2024-01-16T14:54:45.867265+0000","last_active":"2024-01-16T14:54:46.854240+0000","last_peered":"2024-01-16T14:54:46.854240+0000","last_clean":"2024-01-16T14:54:46.854240+0000","last_became_active":"2024-01-16T14:54:45.865756+0000","last_became_peered":"2024-01-16T14:54:45.865756+0000","last_unstale":"2024-01-16T14:54:46.854240+0000","last_undegraded":"2024-01-16T14:54:46.854240+0000","last_fullsized":"2024-01-16T14:54:46.854240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854088+0000","last_change":"2024-01-16T14:54:44.853662+0000","last_active":"2024-01-16T14:54:46.854088+0000","last_peered":"2024-01-16T14:54:46.854088+0000","last_clean":"2024-01-16T14:54:46.854088+0000","last_became_active":"2024-01-16T14:54:44.853379+0000","last_became_peered":"2024-01-16T14:54:44.853379+0000","last_unstale":"2024-01-16T14:54:46.854088+0000","last_undegraded":"2024-01-16T14:54:46.854088+0000","last_fullsized":"2024-01-16T14:54:46.854088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262160+0000","last_change":"2024-01-16T14:54:45.880017+0000","last_active":"2024-01-16T14:54:47.262160+0000","last_peered":"2024-01-16T14:54:47.262160+0000","last_clean":"2024-01-16T14:54:47.262160+0000","last_became_active":"2024-01-16T14:54:45.879369+0000","last_became_peered":"2024-01-16T14:54:45.879369+0000","last_unstale":"2024-01-16T14:54:47.262160+0000","last_undegraded":"2024-01-16T14:54:47.262160+0000","last_fullsized":"2024-01-16T14:54:47.262160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853963+0000","last_change":"2024-01-16T14:54:44.852663+0000","last_active":"2024-01-16T14:54:46.853963+0000","last_peered":"2024-01-16T14:54:46.853963+0000","last_clean":"2024-01-16T14:54:46.853963+0000","last_became_active":"2024-01-16T14:54:44.852346+0000","last_became_peered":"2024-01-16T14:54:44.852346+0000","last_unstale":"2024-01-16T14:54:46.853963+0000","last_undegraded":"2024-01-16T14:54:46.853963+0000","last_fullsized":"2024-01-16T14:54:46.853963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:54.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261811+0000","last_change":"2024-01-16T14:54:45.886559+0000","last_active":"2024-01-16T14:54:47.261811+0000","last_peered":"2024-01-16T14:54:47.261811+0000","last_clean":"2024-01-16T14:54:47.261811+0000","last_became_active":"2024-01-16T14:54:45.886343+0000","last_became_peered":"2024-01-16T14:54:45.886343+0000","last_unstale":"2024-01-16T14:54:47.261811+0000","last_undegraded":"2024-01-16T14:54:47.261811+0000","last_fullsized":"2024-01-16T14:54:47.261811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853331+0000","last_change":"2024-01-16T14:54:44.852311+0000","last_active":"2024-01-16T14:54:46.853331+0000","last_peered":"2024-01-16T14:54:46.853331+0000","last_clean":"2024-01-16T14:54:46.853331+0000","last_became_active":"2024-01-16T14:54:44.852018+0000","last_became_peered":"2024-01-16T14:54:44.852018+0000","last_unstale":"2024-01-16T14:54:46.853331+0000","last_undegraded":"2024-01-16T14:54:46.853331+0000","last_fullsized":"2024-01-16T14:54:46.853331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262110+0000","last_change":"2024-01-16T14:54:45.883384+0000","last_active":"2024-01-16T14:54:47.262110+0000","last_peered":"2024-01-16T14:54:47.262110+0000","last_clean":"2024-01-16T14:54:47.262110+0000","last_became_active":"2024-01-16T14:54:45.883094+0000","last_became_peered":"2024-01-16T14:54:45.883094+0000","last_unstale":"2024-01-16T14:54:47.262110+0000","last_undegraded":"2024-01-16T14:54:47.262110+0000","last_fullsized":"2024-01-16T14:54:47.262110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853863+0000","last_change":"2024-01-16T14:54:44.846655+0000","last_active":"2024-01-16T14:54:46.853863+0000","last_peered":"2024-01-16T14:54:46.853863+0000","last_clean":"2024-01-16T14:54:46.853863+0000","last_became_active":"2024-01-16T14:54:44.846359+0000","last_became_peered":"2024-01-16T14:54:44.846359+0000","last_unstale":"2024-01-16T14:54:46.853863+0000","last_undegraded":"2024-01-16T14:54:46.853863+0000","last_fullsized":"2024-01-16T14:54:46.853863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:28.971339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854823+0000","last_change":"2024-01-16T14:54:45.857956+0000","last_active":"2024-01-16T14:54:46.854823+0000","last_peered":"2024-01-16T14:54:46.854823+0000","last_clean":"2024-01-16T14:54:46.854823+0000","last_became_active":"2024-01-16T14:54:45.857645+0000","last_became_peered":"2024-01-16T14:54:45.857645+0000","last_unstale":"2024-01-16T14:54:46.854823+0000","last_undegraded":"2024-01-16T14:54:46.854823+0000","last_fullsized":"2024-01-16T14:54:46.854823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:39.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261584+0000","last_change":"2024-01-16T14:54:44.851910+0000","last_active":"2024-01-16T14:54:47.261584+0000","last_peered":"2024-01-16T14:54:47.261584+0000","last_clean":"2024-01-16T14:54:47.261584+0000","last_became_active":"2024-01-16T14:54:44.851631+0000","last_became_peered":"2024-01-16T14:54:44.851631+0000","last_unstale":"2024-01-16T14:54:47.261584+0000","last_undegraded":"2024-01-16T14:54:47.261584+0000","last_fullsized":"2024-01-16T14:54:47.261584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:17.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262365+0000","last_change":"2024-01-16T14:54:45.875708+0000","last_active":"2024-01-16T14:54:47.262365+0000","last_peered":"2024-01-16T14:54:47.262365+0000","last_clean":"2024-01-16T14:54:47.262365+0000","last_became_active":"2024-01-16T14:54:45.875328+0000","last_became_peered":"2024-01-16T14:54:45.875328+0000","last_unstale":"2024-01-16T14:54:47.262365+0000","last_undegraded":"2024-01-16T14:54:47.262365+0000","last_fullsized":"2024-01-16T14:54:47.262365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853648+0000","last_change":"2024-01-16T14:54:44.855286+0000","last_active":"2024-01-16T14:54:46.853648+0000","last_peered":"2024-01-16T14:54:46.853648+0000","last_clean":"2024-01-16T14:54:46.853648+0000","last_became_active":"2024-01-16T14:54:44.854465+0000","last_became_peered":"2024-01-16T14:54:44.854465+0000","last_unstale":"2024-01-16T14:54:46.853648+0000","last_undegraded":"2024-01-16T14:54:46.853648+0000","last_fullsized":"2024-01-16T14:54:46.853648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853759+0000","last_change":"2024-01-16T14:54:45.868676+0000","last_active":"2024-01-16T14:54:46.853759+0000","last_peered":"2024-01-16T14:54:46.853759+0000","last_clean":"2024-01-16T14:54:46.853759+0000","last_became_active":"2024-01-16T14:54:45.868354+0000","last_became_peered":"2024-01-16T14:54:45.868354+0000","last_unstale":"2024-01-16T14:54:46.853759+0000","last_undegraded":"2024-01-16T14:54:46.853759+0000","last_fullsized":"2024-01-16T14:54:46.853759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261155+0000","last_change":"2024-01-16T14:54:44.855323+0000","last_active":"2024-01-16T14:54:47.261155+0000","last_peered":"2024-01-16T14:54:47.261155+0000","last_clean":"2024-01-16T14:54:47.261155+0000","last_became_active":"2024-01-16T14:54:44.855035+0000","last_became_peered":"2024-01-16T14:54:44.855035+0000","last_unstale":"2024-01-16T14:54:47.261155+0000","last_undegraded":"2024-01-16T14:54:47.261155+0000","last_fullsized":"2024-01-16T14:54:47.261155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:33.068253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262370+0000","last_change":"2024-01-16T14:54:45.864396+0000","last_active":"2024-01-16T14:54:47.262370+0000","last_peered":"2024-01-16T14:54:47.262370+0000","last_clean":"2024-01-16T14:54:47.262370+0000","last_became_active":"2024-01-16T14:54:45.864052+0000","last_became_peered":"2024-01-16T14:54:45.864052+0000","last_unstale":"2024-01-16T14:54:47.262370+0000","last_undegraded":"2024-01-16T14:54:47.262370+0000","last_fullsized":"2024-01-16T14:54:47.262370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262332+0000","last_change":"2024-01-16T14:54:44.857192+0000","last_active":"2024-01-16T14:54:47.262332+0000","last_peered":"2024-01-16T14:54:47.262332+0000","last_clean":"2024-01-16T14:54:47.262332+0000","last_became_active":"2024-01-16T14:54:44.856880+0000","last_became_peered":"2024-01-16T14:54:44.856880+0000","last_unstale":"2024-01-16T14:54:47.262332+0000","last_undegraded":"2024-01-16T14:54:47.262332+0000","last_fullsized":"2024-01-16T14:54:47.262332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261868+0000","last_change":"2024-01-16T14:54:45.884208+0000","last_active":"2024-01-16T14:54:47.261868+0000","last_peered":"2024-01-16T14:54:47.261868+0000","last_clean":"2024-01-16T14:54:47.261868+0000","last_became_active":"2024-01-16T14:54:45.882540+0000","last_became_peered":"2024-01-16T14:54:45.882540+0000","last_unstale":"2024-01-16T14:54:47.261868+0000","last_undegraded":"2024-01-16T14:54:47.261868+0000","last_fullsized":"2024-01-16T14:54:47.261868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261721+0000","last_change":"2024-01-16T14:54:44.854856+0000","last_active":"2024-01-16T14:54:47.261721+0000","last_peered":"2024-01-16T14:54:47.261721+0000","last_clean":"2024-01-16T14:54:47.261721+0000","last_became_active":"2024-01-16T14:54:44.854543+0000","last_became_peered":"2024-01-16T14:54:44.854543+0000","last_unstale":"2024-01-16T14:54:47.261721+0000","last_undegraded":"2024-01-16T14:54:47.261721+0000","last_fullsized":"2024-01-16T14:54:47.261721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262312+0000","last_change":"2024-01-16T14:54:45.880240+0000","last_active":"2024-01-16T14:54:47.262312+0000","last_peered":"2024-01-16T14:54:47.262312+0000","last_clean":"2024-01-16T14:54:47.262312+0000","last_became_active":"2024-01-16T14:54:45.879764+0000","last_became_peered":"2024-01-16T14:54:45.879764+0000","last_unstale":"2024-01-16T14:54:47.262312+0000","last_undegraded":"2024-01-16T14:54:47.262312+0000","last_fullsized":"2024-01-16T14:54:47.262312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262763+0000","last_change":"2024-01-16T14:54:44.855449+0000","last_active":"2024-01-16T14:54:47.262763+0000","last_peered":"2024-01-16T14:54:47.262763+0000","last_clean":"2024-01-16T14:54:47.262763+0000","last_became_active":"2024-01-16T14:54:44.855037+0000","last_became_peered":"2024-01-16T14:54:44.855037+0000","last_unstale":"2024-01-16T14:54:47.262763+0000","last_undegraded":"2024-01-16T14:54:47.262763+0000","last_fullsized":"2024-01-16T14:54:47.262763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855392+0000","last_change":"2024-01-16T14:54:45.870932+0000","last_active":"2024-01-16T14:54:46.855392+0000","last_peered":"2024-01-16T14:54:46.855392+0000","last_clean":"2024-01-16T14:54:46.855392+0000","last_became_active":"2024-01-16T14:54:45.870016+0000","last_became_peered":"2024-01-16T14:54:45.870016+0000","last_unstale":"2024-01-16T14:54:46.855392+0000","last_undegraded":"2024-01-16T14:54:46.855392+0000","last_fullsized":"2024-01-16T14:54:46.855392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261353+0000","last_change":"2024-01-16T14:54:44.854762+0000","last_active":"2024-01-16T14:54:47.261353+0000","last_peered":"2024-01-16T14:54:47.261353+0000","last_clean":"2024-01-16T14:54:47.261353+0000","last_became_active":"2024-01-16T14:54:44.854507+0000","last_became_peered":"2024-01-16T14:54:44.854507+0000","last_unstale":"2024-01-16T14:54:47.261353+0000","last_undegraded":"2024-01-16T14:54:47.261353+0000","last_fullsized":"2024-01-16T14:54:47.261353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853580+0000","last_change":"2024-01-16T14:54:45.879814+0000","last_active":"2024-01-16T14:54:46.853580+0000","last_peered":"2024-01-16T14:54:46.853580+0000","last_clean":"2024-01-16T14:54:46.853580+0000","last_became_active":"2024-01-16T14:54:45.873897+0000","last_became_peered":"2024-01-16T14:54:45.873897+0000","last_unstale":"2024-01-16T14:54:46.853580+0000","last_undegraded":"2024-01-16T14:54:46.853580+0000","last_fullsized":"2024-01-16T14:54:46.853580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262205+0000","last_change":"2024-01-16T14:54:44.860218+0000","last_active":"2024-01-16T14:54:47.262205+0000","last_peered":"2024-01-16T14:54:47.262205+0000","last_clean":"2024-01-16T14:54:47.262205+0000","last_became_active":"2024-01-16T14:54:44.859920+0000","last_became_peered":"2024-01-16T14:54:44.859920+0000","last_unstale":"2024-01-16T14:54:47.262205+0000","last_undegraded":"2024-01-16T14:54:47.262205+0000","last_fullsized":"2024-01-16T14:54:47.262205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262345+0000","last_change":"2024-01-16T14:54:45.875405+0000","last_active":"2024-01-16T14:54:47.262345+0000","last_peered":"2024-01-16T14:54:47.262345+0000","last_clean":"2024-01-16T14:54:47.262345+0000","last_became_active":"2024-01-16T14:54:45.874428+0000","last_became_peered":"2024-01-16T14:54:45.874428+0000","last_unstale":"2024-01-16T14:54:47.262345+0000","last_undegraded":"2024-01-16T14:54:47.262345+0000","last_fullsized":"2024-01-16T14:54:47.262345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:27.912278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262076+0000","last_change":"2024-01-16T14:54:44.857601+0000","last_active":"2024-01-16T14:54:47.262076+0000","last_peered":"2024-01-16T14:54:47.262076+0000","last_clean":"2024-01-16T14:54:47.262076+0000","last_became_active":"2024-01-16T14:54:44.857282+0000","last_became_peered":"2024-01-16T14:54:44.857282+0000","last_unstale":"2024-01-16T14:54:47.262076+0000","last_undegraded":"2024-01-16T14:54:47.262076+0000","last_fullsized":"2024-01-16T14:54:47.262076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853602+0000","last_change":"2024-01-16T14:54:45.873626+0000","last_active":"2024-01-16T14:54:46.853602+0000","last_peered":"2024-01-16T14:54:46.853602+0000","last_clean":"2024-01-16T14:54:46.853602+0000","last_became_active":"2024-01-16T14:54:45.872632+0000","last_became_peered":"2024-01-16T14:54:45.872632+0000","last_unstale":"2024-01-16T14:54:46.853602+0000","last_undegraded":"2024-01-16T14:54:46.853602+0000","last_fullsized":"2024-01-16T14:54:46.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261375+0000","last_change":"2024-01-16T14:54:44.852946+0000","last_active":"2024-01-16T14:54:47.261375+0000","last_peered":"2024-01-16T14:54:47.261375+0000","last_clean":"2024-01-16T14:54:47.261375+0000","last_became_active":"2024-01-16T14:54:44.852682+0000","last_became_peered":"2024-01-16T14:54:44.852682+0000","last_unstale":"2024-01-16T14:54:47.261375+0000","last_undegraded":"2024-01-16T14:54:47.261375+0000","last_fullsized":"2024-01-16T14:54:47.261375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261947+0000","last_change":"2024-01-16T14:54:45.877152+0000","last_active":"2024-01-16T14:54:47.261947+0000","last_peered":"2024-01-16T14:54:47.261947+0000","last_clean":"2024-01-16T14:54:47.261947+0000","last_became_active":"2024-01-16T14:54:45.876716+0000","last_became_peered":"2024-01-16T14:54:45.876716+0000","last_unstale":"2024-01-16T14:54:47.261947+0000","last_undegraded":"2024-01-16T14:54:47.261947+0000","last_fullsized":"2024-01-16T14:54:47.261947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261401+0000","last_change":"2024-01-16T14:54:44.857725+0000","last_active":"2024-01-16T14:54:47.261401+0000","last_peered":"2024-01-16T14:54:47.261401+0000","last_clean":"2024-01-16T14:54:47.261401+0000","last_became_active":"2024-01-16T14:54:44.857273+0000","last_became_peered":"2024-01-16T14:54:44.857273+0000","last_unstale":"2024-01-16T14:54:47.261401+0000","last_undegraded":"2024-01-16T14:54:47.261401+0000","last_fullsized":"2024-01-16T14:54:47.261401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853385+0000","last_change":"2024-01-16T14:54:45.868095+0000","last_active":"2024-01-16T14:54:46.853385+0000","last_peered":"2024-01-16T14:54:46.853385+0000","last_clean":"2024-01-16T14:54:46.853385+0000","last_became_active":"2024-01-16T14:54:45.867008+0000","last_became_peered":"2024-01-16T14:54:45.867008+0000","last_unstale":"2024-01-16T14:54:46.853385+0000","last_undegraded":"2024-01-16T14:54:46.853385+0000","last_fullsized":"2024-01-16T14:54:46.853385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855353+0000","last_change":"2024-01-16T14:54:44.853674+0000","last_active":"2024-01-16T14:54:46.855353+0000","last_peered":"2024-01-16T14:54:46.855353+0000","last_clean":"2024-01-16T14:54:46.855353+0000","last_became_active":"2024-01-16T14:54:44.853079+0000","last_became_peered":"2024-01-16T14:54:44.853079+0000","last_unstale":"2024-01-16T14:54:46.855353+0000","last_undegraded":"2024-01-16T14:54:46.855353+0000","last_fullsized":"2024-01-16T14:54:46.855353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:31.747933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853861+0000","last_change":"2024-01-16T14:54:45.875678+0000","last_active":"2024-01-16T14:54:46.853861+0000","last_peered":"2024-01-16T14:54:46.853861+0000","last_clean":"2024-01-16T14:54:46.853861+0000","last_became_active":"2024-01-16T14:54:45.875400+0000","last_became_peered":"2024-01-16T14:54:45.875400+0000","last_unstale":"2024-01-16T14:54:46.853861+0000","last_undegraded":"2024-01-16T14:54:46.853861+0000","last_fullsized":"2024-01-16T14:54:46.853861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262644+0000","last_change":"2024-01-16T14:54:44.858592+0000","last_active":"2024-01-16T14:54:47.262644+0000","last_peered":"2024-01-16T14:54:47.262644+0000","last_clean":"2024-01-16T14:54:47.262644+0000","last_became_active":"2024-01-16T14:54:44.858275+0000","last_became_peered":"2024-01-16T14:54:44.858275+0000","last_unstale":"2024-01-16T14:54:47.262644+0000","last_undegraded":"2024-01-16T14:54:47.262644+0000","last_fullsized":"2024-01-16T14:54:47.262644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262293+0000","last_change":"2024-01-16T14:54:45.877869+0000","last_active":"2024-01-16T14:54:47.262293+0000","last_peered":"2024-01-16T14:54:47.262293+0000","last_clean":"2024-01-16T14:54:47.262293+0000","last_became_active":"2024-01-16T14:54:45.876905+0000","last_became_peered":"2024-01-16T14:54:45.876905+0000","last_unstale":"2024-01-16T14:54:47.262293+0000","last_undegraded":"2024-01-16T14:54:47.262293+0000","last_fullsized":"2024-01-16T14:54:47.262293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:47:51.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261976+0000","last_change":"2024-01-16T14:54:44.855986+0000","last_active":"2024-01-16T14:54:47.261976+0000","last_peered":"2024-01-16T14:54:47.261976+0000","last_clean":"2024-01-16T14:54:47.261976+0000","last_became_active":"2024-01-16T14:54:44.855690+0000","last_became_peered":"2024-01-16T14:54:44.855690+0000","last_unstale":"2024-01-16T14:54:47.261976+0000","last_undegraded":"2024-01-16T14:54:47.261976+0000","last_fullsized":"2024-01-16T14:54:47.261976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855013+0000","last_change":"2024-01-16T14:54:45.878441+0000","last_active":"2024-01-16T14:54:46.855013+0000","last_peered":"2024-01-16T14:54:46.855013+0000","last_clean":"2024-01-16T14:54:46.855013+0000","last_became_active":"2024-01-16T14:54:45.878011+0000","last_became_peered":"2024-01-16T14:54:45.878011+0000","last_unstale":"2024-01-16T14:54:46.855013+0000","last_undegraded":"2024-01-16T14:54:46.855013+0000","last_fullsized":"2024-01-16T14:54:46.855013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:05.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261593+0000","last_change":"2024-01-16T14:54:45.869056+0000","last_active":"2024-01-16T14:54:47.261593+0000","last_peered":"2024-01-16T14:54:47.261593+0000","last_clean":"2024-01-16T14:54:47.261593+0000","last_became_active":"2024-01-16T14:54:45.868748+0000","last_became_peered":"2024-01-16T14:54:45.868748+0000","last_unstale":"2024-01-16T14:54:47.261593+0000","last_undegraded":"2024-01-16T14:54:47.261593+0000","last_fullsized":"2024-01-16T14:54:47.261593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261832+0000","last_change":"2024-01-16T14:54:44.853961+0000","last_active":"2024-01-16T14:54:47.261832+0000","last_peered":"2024-01-16T14:54:47.261832+0000","last_clean":"2024-01-16T14:54:47.261832+0000","last_became_active":"2024-01-16T14:54:44.853610+0000","last_became_peered":"2024-01-16T14:54:44.853610+0000","last_unstale":"2024-01-16T14:54:47.261832+0000","last_undegraded":"2024-01-16T14:54:47.261832+0000","last_fullsized":"2024-01-16T14:54:47.261832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853929+0000","last_change":"2024-01-16T14:54:45.872254+0000","last_active":"2024-01-16T14:54:46.853929+0000","last_peered":"2024-01-16T14:54:46.853929+0000","last_clean":"2024-01-16T14:54:46.853929+0000","last_became_active":"2024-01-16T14:54:45.870597+0000","last_became_peered":"2024-01-16T14:54:45.870597+0000","last_unstale":"2024-01-16T14:54:46.853929+0000","last_undegraded":"2024-01-16T14:54:46.853929+0000","last_fullsized":"2024-01-16T14:54:46.853929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261437+0000","last_change":"2024-01-16T14:54:44.854781+0000","last_active":"2024-01-16T14:54:47.261437+0000","last_peered":"2024-01-16T14:54:47.261437+0000","last_clean":"2024-01-16T14:54:47.261437+0000","last_became_active":"2024-01-16T14:54:44.854461+0000","last_became_peered":"2024-01-16T14:54:44.854461+0000","last_unstale":"2024-01-16T14:54:47.261437+0000","last_undegraded":"2024-01-16T14:54:47.261437+0000","last_fullsized":"2024-01-16T14:54:47.261437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T14:54:45.859524+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:54:45.859524+0000","last_peered":"2024-01-16T14:54:45.859524+0000","last_clean":"2024-01-16T14:54:45.859524+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:54:45.859524+0000","last_undegraded":"2024-01-16T14:54:45.859524+0000","last_fullsized":"2024-01-16T14:54:45.859524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261727+0000","last_change":"2024-01-16T14:54:45.875199+0000","last_active":"2024-01-16T14:54:47.261727+0000","last_peered":"2024-01-16T14:54:47.261727+0000","last_clean":"2024-01-16T14:54:47.261727+0000","last_became_active":"2024-01-16T14:54:45.874049+0000","last_became_peered":"2024-01-16T14:54:45.874049+0000","last_unstale":"2024-01-16T14:54:47.261727+0000","last_undegraded":"2024-01-16T14:54:47.261727+0000","last_fullsized":"2024-01-16T14:54:47.261727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:29.976614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853314+0000","last_change":"2024-01-16T14:54:44.857691+0000","last_active":"2024-01-16T14:54:46.853314+0000","last_peered":"2024-01-16T14:54:46.853314+0000","last_clean":"2024-01-16T14:54:46.853314+0000","last_became_active":"2024-01-16T14:54:44.857407+0000","last_became_peered":"2024-01-16T14:54:44.857407+0000","last_unstale":"2024-01-16T14:54:46.853314+0000","last_undegraded":"2024-01-16T14:54:46.853314+0000","last_fullsized":"2024-01-16T14:54:46.853314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853856+0000","last_change":"2024-01-16T14:54:44.853542+0000","last_active":"2024-01-16T14:54:46.853856+0000","last_peered":"2024-01-16T14:54:46.853856+0000","last_clean":"2024-01-16T14:54:46.853856+0000","last_became_active":"2024-01-16T14:54:44.853273+0000","last_became_peered":"2024-01-16T14:54:44.853273+0000","last_unstale":"2024-01-16T14:54:46.853856+0000","last_undegraded":"2024-01-16T14:54:46.853856+0000","last_fullsized":"2024-01-16T14:54:46.853856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:56.322689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262718+0000","last_change":"2024-01-16T14:54:45.868977+0000","last_active":"2024-01-16T14:54:47.262718+0000","last_peered":"2024-01-16T14:54:47.262718+0000","last_clean":"2024-01-16T14:54:47.262718+0000","last_became_active":"2024-01-16T14:54:45.868675+0000","last_became_peered":"2024-01-16T14:54:45.868675+0000","last_unstale":"2024-01-16T14:54:47.262718+0000","last_undegraded":"2024-01-16T14:54:47.262718+0000","last_fullsized":"2024-01-16T14:54:47.262718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262724+0000","last_change":"2024-01-16T14:54:44.855466+0000","last_active":"2024-01-16T14:54:47.262724+0000","last_peered":"2024-01-16T14:54:47.262724+0000","last_clean":"2024-01-16T14:54:47.262724+0000","last_became_active":"2024-01-16T14:54:44.855144+0000","last_became_peered":"2024-01-16T14:54:44.855144+0000","last_unstale":"2024-01-16T14:54:47.262724+0000","last_undegraded":"2024-01-16T14:54:47.262724+0000","last_fullsized":"2024-01-16T14:54:47.262724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854544+0000","last_change":"2024-01-16T14:54:45.875954+0000","last_active":"2024-01-16T14:54:46.854544+0000","last_peered":"2024-01-16T14:54:46.854544+0000","last_clean":"2024-01-16T14:54:46.854544+0000","last_became_active":"2024-01-16T14:54:45.875635+0000","last_became_peered":"2024-01-16T14:54:45.875635+0000","last_unstale":"2024-01-16T14:54:46.854544+0000","last_undegraded":"2024-01-16T14:54:46.854544+0000","last_fullsized":"2024-01-16T14:54:46.854544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262344+0000","last_change":"2024-01-16T14:54:44.858826+0000","last_active":"2024-01-16T14:54:47.262344+0000","last_peered":"2024-01-16T14:54:47.262344+0000","last_clean":"2024-01-16T14:54:47.262344+0000","last_became_active":"2024-01-16T14:54:44.858528+0000","last_became_peered":"2024-01-16T14:54:44.858528+0000","last_unstale":"2024-01-16T14:54:47.262344+0000","last_undegraded":"2024-01-16T14:54:47.262344+0000","last_fullsized":"2024-01-16T14:54:47.262344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262074+0000","last_change":"2024-01-16T14:54:45.872205+0000","last_active":"2024-01-16T14:54:47.262074+0000","last_peered":"2024-01-16T14:54:47.262074+0000","last_clean":"2024-01-16T14:54:47.262074+0000","last_became_active":"2024-01-16T14:54:45.871893+0000","last_became_peered":"2024-01-16T14:54:45.871893+0000","last_unstale":"2024-01-16T14:54:47.262074+0000","last_undegraded":"2024-01-16T14:54:47.262074+0000","last_fullsized":"2024-01-16T14:54:47.262074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853147+0000","last_change":"2024-01-16T14:54:44.853929+0000","last_active":"2024-01-16T14:54:46.853147+0000","last_peered":"2024-01-16T14:54:46.853147+0000","last_clean":"2024-01-16T14:54:46.853147+0000","last_became_active":"2024-01-16T14:54:44.853655+0000","last_became_peered":"2024-01-16T14:54:44.853655+0000","last_unstale":"2024-01-16T14:54:46.853147+0000","last_undegraded":"2024-01-16T14:54:46.853147+0000","last_fullsized":"2024-01-16T14:54:46.853147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855031+0000","last_change":"2024-01-16T14:54:45.870957+0000","last_active":"2024-01-16T14:54:46.855031+0000","last_peered":"2024-01-16T14:54:46.855031+0000","last_clean":"2024-01-16T14:54:46.855031+0000","last_became_active":"2024-01-16T14:54:45.870672+0000","last_became_peered":"2024-01-16T14:54:45.870672+0000","last_unstale":"2024-01-16T14:54:46.855031+0000","last_undegraded":"2024-01-16T14:54:46.855031+0000","last_fullsized":"2024-01-16T14:54:46.855031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:19.153610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261884+0000","last_change":"2024-01-16T14:54:44.859174+0000","last_active":"2024-01-16T14:54:47.261884+0000","last_peered":"2024-01-16T14:54:47.261884+0000","last_clean":"2024-01-16T14:54:47.261884+0000","last_became_active":"2024-01-16T14:54:44.858845+0000","last_became_peered":"2024-01-16T14:54:44.858845+0000","last_unstale":"2024-01-16T14:54:47.261884+0000","last_undegraded":"2024-01-16T14:54:47.261884+0000","last_fullsized":"2024-01-16T14:54:47.261884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853729+0000","last_change":"2024-01-16T14:54:45.865608+0000","last_active":"2024-01-16T14:54:46.853729+0000","last_peered":"2024-01-16T14:54:46.853729+0000","last_clean":"2024-01-16T14:54:46.853729+0000","last_became_active":"2024-01-16T14:54:45.864856+0000","last_became_peered":"2024-01-16T14:54:45.864856+0000","last_unstale":"2024-01-16T14:54:46.853729+0000","last_undegraded":"2024-01-16T14:54:46.853729+0000","last_fullsized":"2024-01-16T14:54:46.853729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262559+0000","last_change":"2024-01-16T14:54:44.857846+0000","last_active":"2024-01-16T14:54:47.262559+0000","last_peered":"2024-01-16T14:54:47.262559+0000","last_clean":"2024-01-16T14:54:47.262559+0000","last_became_active":"2024-01-16T14:54:44.857244+0000","last_became_peered":"2024-01-16T14:54:44.857244+0000","last_unstale":"2024-01-16T14:54:47.262559+0000","last_undegraded":"2024-01-16T14:54:47.262559+0000","last_fullsized":"2024-01-16T14:54:47.262559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854055+0000","last_change":"2024-01-16T14:54:45.877240+0000","last_active":"2024-01-16T14:54:46.854055+0000","last_peered":"2024-01-16T14:54:46.854055+0000","last_clean":"2024-01-16T14:54:46.854055+0000","last_became_active":"2024-01-16T14:54:45.876921+0000","last_became_peered":"2024-01-16T14:54:45.876921+0000","last_unstale":"2024-01-16T14:54:46.854055+0000","last_undegraded":"2024-01-16T14:54:46.854055+0000","last_fullsized":"2024-01-16T14:54:46.854055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853720+0000","last_change":"2024-01-16T14:54:44.841886+0000","last_active":"2024-01-16T14:54:46.853720+0000","last_peered":"2024-01-16T14:54:46.853720+0000","last_clean":"2024-01-16T14:54:46.853720+0000","last_became_active":"2024-01-16T14:54:44.841590+0000","last_became_peered":"2024-01-16T14:54:44.841590+0000","last_unstale":"2024-01-16T14:54:46.853720+0000","last_undegraded":"2024-01-16T14:54:46.853720+0000","last_fullsized":"2024-01-16T14:54:46.853720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261806+0000","last_change":"2024-01-16T14:54:45.861293+0000","last_active":"2024-01-16T14:54:47.261806+0000","last_peered":"2024-01-16T14:54:47.261806+0000","last_clean":"2024-01-16T14:54:47.261806+0000","last_became_active":"2024-01-16T14:54:45.860966+0000","last_became_peered":"2024-01-16T14:54:45.860966+0000","last_unstale":"2024-01-16T14:54:47.261806+0000","last_undegraded":"2024-01-16T14:54:47.261806+0000","last_fullsized":"2024-01-16T14:54:47.261806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262495+0000","last_change":"2024-01-16T14:54:44.859596+0000","last_active":"2024-01-16T14:54:47.262495+0000","last_peered":"2024-01-16T14:54:47.262495+0000","last_clean":"2024-01-16T14:54:47.262495+0000","last_became_active":"2024-01-16T14:54:44.859150+0000","last_became_peered":"2024-01-16T14:54:44.859150+0000","last_unstale":"2024-01-16T14:54:47.262495+0000","last_undegraded":"2024-01-16T14:54:47.262495+0000","last_fullsized":"2024-01-16T14:54:47.262495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853762+0000","last_change":"2024-01-16T14:54:45.884933+0000","last_active":"2024-01-16T14:54:46.853762+0000","last_peered":"2024-01-16T14:54:46.853762+0000","last_clean":"2024-01-16T14:54:46.853762+0000","last_became_active":"2024-01-16T14:54:45.884688+0000","last_became_peered":"2024-01-16T14:54:45.884688+0000","last_unstale":"2024-01-16T14:54:46.853762+0000","last_undegraded":"2024-01-16T14:54:46.853762+0000","last_fullsized":"2024-01-16T14:54:46.853762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262635+0000","last_change":"2024-01-16T14:54:44.852405+0000","last_active":"2024-01-16T14:54:47.262635+0000","last_peered":"2024-01-16T14:54:47.262635+0000","last_clean":"2024-01-16T14:54:47.262635+0000","last_became_active":"2024-01-16T14:54:44.852011+0000","last_became_peered":"2024-01-16T14:54:44.852011+0000","last_unstale":"2024-01-16T14:54:47.262635+0000","last_undegraded":"2024-01-16T14:54:47.262635+0000","last_fullsized":"2024-01-16T14:54:47.262635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261800+0000","last_change":"2024-01-16T14:54:45.883630+0000","last_active":"2024-01-16T14:54:47.261800+0000","last_peered":"2024-01-16T14:54:47.261800+0000","last_clean":"2024-01-16T14:54:47.261800+0000","last_became_active":"2024-01-16T14:54:45.882901+0000","last_became_peered":"2024-01-16T14:54:45.882901+0000","last_unstale":"2024-01-16T14:54:47.261800+0000","last_undegraded":"2024-01-16T14:54:47.261800+0000","last_fullsized":"2024-01-16T14:54:47.261800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262954+0000","last_change":"2024-01-16T14:54:44.853825+0000","last_active":"2024-01-16T14:54:47.262954+0000","last_peered":"2024-01-16T14:54:47.262954+0000","last_clean":"2024-01-16T14:54:47.262954+0000","last_became_active":"2024-01-16T14:54:44.853509+0000","last_became_peered":"2024-01-16T14:54:44.853509+0000","last_unstale":"2024-01-16T14:54:47.262954+0000","last_undegraded":"2024-01-16T14:54:47.262954+0000","last_fullsized":"2024-01-16T14:54:47.262954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853960+0000","last_change":"2024-01-16T14:54:45.878029+0000","last_active":"2024-01-16T14:54:46.853960+0000","last_peered":"2024-01-16T14:54:46.853960+0000","last_clean":"2024-01-16T14:54:46.853960+0000","last_became_active":"2024-01-16T14:54:45.877266+0000","last_became_peered":"2024-01-16T14:54:45.877266+0000","last_unstale":"2024-01-16T14:54:46.853960+0000","last_undegraded":"2024-01-16T14:54:46.853960+0000","last_fullsized":"2024-01-16T14:54:46.853960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261228+0000","last_change":"2024-01-16T14:54:45.876962+0000","last_active":"2024-01-16T14:54:47.261228+0000","last_peered":"2024-01-16T14:54:47.261228+0000","last_clean":"2024-01-16T14:54:47.261228+0000","last_became_active":"2024-01-16T14:54:45.876358+0000","last_became_peered":"2024-01-16T14:54:45.876358+0000","last_unstale":"2024-01-16T14:54:47.261228+0000","last_undegraded":"2024-01-16T14:54:47.261228+0000","last_fullsized":"2024-01-16T14:54:47.261228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261492+0000","last_change":"2024-01-16T14:54:45.874788+0000","last_active":"2024-01-16T14:54:47.261492+0000","last_peered":"2024-01-16T14:54:47.261492+0000","last_clean":"2024-01-16T14:54:47.261492+0000","last_became_active":"2024-01-16T14:54:45.874453+0000","last_became_peered":"2024-01-16T14:54:45.874453+0000","last_unstale":"2024-01-16T14:54:47.261492+0000","last_undegraded":"2024-01-16T14:54:47.261492+0000","last_fullsized":"2024-01-16T14:54:47.261492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261754+0000","last_change":"2024-01-16T14:54:45.877374+0000","last_active":"2024-01-16T14:54:47.261754+0000","last_peered":"2024-01-16T14:54:47.261754+0000","last_clean":"2024-01-16T14:54:47.261754+0000","last_became_active":"2024-01-16T14:54:45.877118+0000","last_became_peered":"2024-01-16T14:54:45.877118+0000","last_unstale":"2024-01-16T14:54:47.261754+0000","last_undegraded":"2024-01-16T14:54:47.261754+0000","last_fullsized":"2024-01-16T14:54:47.261754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262220+0000","last_change":"2024-01-16T14:54:45.875476+0000","last_active":"2024-01-16T14:54:47.262220+0000","last_peered":"2024-01-16T14:54:47.262220+0000","last_clean":"2024-01-16T14:54:47.262220+0000","last_became_active":"2024-01-16T14:54:45.874303+0000","last_became_peered":"2024-01-16T14:54:45.874303+0000","last_unstale":"2024-01-16T14:54:47.262220+0000","last_undegraded":"2024-01-16T14:54:47.262220+0000","last_fullsized":"2024-01-16T14:54:47.262220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854023+0000","last_change":"2024-01-16T14:54:45.866307+0000","last_active":"2024-01-16T14:54:46.854023+0000","last_peered":"2024-01-16T14:54:46.854023+0000","last_clean":"2024-01-16T14:54:46.854023+0000","last_became_active":"2024-01-16T14:54:45.865419+0000","last_became_peered":"2024-01-16T14:54:45.865419+0000","last_unstale":"2024-01-16T14:54:46.854023+0000","last_undegraded":"2024-01-16T14:54:46.854023+0000","last_fullsized":"2024-01-16T14:54:46.854023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262723+0000","last_change":"2024-01-16T14:54:45.866420+0000","last_active":"2024-01-16T14:54:47.262723+0000","last_peered":"2024-01-16T14:54:47.262723+0000","last_clean":"2024-01-16T14:54:47.262723+0000","last_became_active":"2024-01-16T14:54:45.866022+0000","last_became_peered":"2024-01-16T14:54:45.866022+0000","last_unstale":"2024-01-16T14:54:47.262723+0000","last_undegraded":"2024-01-16T14:54:47.262723+0000","last_fullsized":"2024-01-16T14:54:47.262723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262181+0000","last_change":"2024-01-16T14:54:45.875523+0000","last_active":"2024-01-16T14:54:47.262181+0000","last_peered":"2024-01-16T14:54:47.262181+0000","last_clean":"2024-01-16T14:54:47.262181+0000","last_became_active":"2024-01-16T14:54:45.875252+0000","last_became_peered":"2024-01-16T14:54:45.875252+0000","last_unstale":"2024-01-16T14:54:47.262181+0000","last_undegraded":"2024-01-16T14:54:47.262181+0000","last_fullsized":"2024-01-16T14:54:47.262181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854401+0000","last_change":"2024-01-16T14:54:45.865807+0000","last_active":"2024-01-16T14:54:46.854401+0000","last_peered":"2024-01-16T14:54:46.854401+0000","last_clean":"2024-01-16T14:54:46.854401+0000","last_became_active":"2024-01-16T14:54:45.865257+0000","last_became_peered":"2024-01-16T14:54:45.865257+0000","last_unstale":"2024-01-16T14:54:46.854401+0000","last_undegraded":"2024-01-16T14:54:46.854401+0000","last_fullsized":"2024-01-16T14:54:46.854401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261310+0000","last_change":"2024-01-16T14:54:45.883032+0000","last_active":"2024-01-16T14:54:47.261310+0000","last_peered":"2024-01-16T14:54:47.261310+0000","last_clean":"2024-01-16T14:54:47.261310+0000","last_became_active":"2024-01-16T14:54:45.882768+0000","last_became_peered":"2024-01-16T14:54:45.882768+0000","last_unstale":"2024-01-16T14:54:47.261310+0000","last_undegraded":"2024-01-16T14:54:47.261310+0000","last_fullsized":"2024-01-16T14:54:47.261310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853768+0000","last_change":"2024-01-16T14:54:45.866167+0000","last_active":"2024-01-16T14:54:46.853768+0000","last_peered":"2024-01-16T14:54:46.853768+0000","last_clean":"2024-01-16T14:54:46.853768+0000","last_became_active":"2024-01-16T14:54:45.865393+0000","last_became_peered":"2024-01-16T14:54:45.865393+0000","last_unstale":"2024-01-16T14:54:46.853768+0000","last_undegraded":"2024-01-16T14:54:46.853768+0000","last_fullsized":"2024-01-16T14:54:46.853768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261572+0000","last_change":"2024-01-16T14:54:45.882612+0000","last_active":"2024-01-16T14:54:47.261572+0000","last_peered":"2024-01-16T14:54:47.261572+0000","last_clean":"2024-01-16T14:54:47.261572+0000","last_became_active":"2024-01-16T14:54:45.882301+0000","last_became_peered":"2024-01-16T14:54:45.882301+0000","last_unstale":"2024-01-16T14:54:47.261572+0000","last_undegraded":"2024-01-16T14:54:47.261572+0000","last_fullsized":"2024-01-16T14:54:47.261572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262090+0000","last_change":"2024-01-16T14:54:45.876292+0000","last_active":"2024-01-16T14:54:47.262090+0000","last_peered":"2024-01-16T14:54:47.262090+0000","last_clean":"2024-01-16T14:54:47.262090+0000","last_became_active":"2024-01-16T14:54:45.875995+0000","last_became_peered":"2024-01-16T14:54:45.875995+0000","last_unstale":"2024-01-16T14:54:47.262090+0000","last_undegraded":"2024-01-16T14:54:47.262090+0000","last_fullsized":"2024-01-16T14:54:47.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261512+0000","last_change":"2024-01-16T14:54:45.866707+0000","last_active":"2024-01-16T14:54:47.261512+0000","last_peered":"2024-01-16T14:54:47.261512+0000","last_clean":"2024-01-16T14:54:47.261512+0000","last_became_active":"2024-01-16T14:54:45.866333+0000","last_became_peered":"2024-01-16T14:54:45.866333+0000","last_unstale":"2024-01-16T14:54:47.261512+0000","last_undegraded":"2024-01-16T14:54:47.261512+0000","last_fullsized":"2024-01-16T14:54:47.261512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:54:50.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:54:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.261+0000 7fa5b4b8b700 1 Processor -- start 2024-01-16T14:54:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.261+0000 7fa5b4b8b700 1 -- start start 2024-01-16T14:54:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5b4b8b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa5b00b57b0 0x7fa5b006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5b4b8b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5b006b510 0x7fa5b006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5b4b8b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b006dbe0 con 0x7fa5b01478c0 2024-01-16T14:54:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5b4b8b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b006b9a0 con 0x7fa5b006b510 2024-01-16T14:54:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5b4b8b700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b0147ca0 con 0x7fa5b00b57b0 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5ae59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5b01478c0 0x7fa5b0161a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58316/0 (socket says 172.21.15.26:58316) 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.262+0000 7fa5ae59c700 1 -- 172.21.15.26:0/828988975 learned_addr learned my addr 172.21.15.26:0/828988975 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 --2- 172.21.15.26:0/828988975 >> v2:172.21.15.26:3300/0 conn(0x7fa5b006b510 0x7fa5b006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5add9b700 1 --2- 172.21.15.26:0/828988975 >> v2:172.21.15.93:3300/0 conn(0x7fa5b00b57b0 0x7fa5b006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5ad59a700 1 -- 172.21.15.26:0/828988975 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027447190 0 0) 0x7fa5b006dbe0 con 0x7fa5b01478c0 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5ad59a700 1 -- 172.21.15.26:0/828988975 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5980035b0 con 0x7fa5b01478c0 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 -- 172.21.15.26:0/828988975 >> v2:172.21.15.93:3300/0 conn(0x7fa5b00b57b0 msgr2=0x7fa5b006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 --2- 172.21.15.26:0/828988975 >> v2:172.21.15.93:3300/0 conn(0x7fa5b00b57b0 0x7fa5b006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 -- 172.21.15.26:0/828988975 >> v1:172.21.15.26:6789/0 conn(0x7fa5b01478c0 legacy=0x7fa5b0161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 -- 172.21.15.26:0/828988975 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b01622e0 con 0x7fa5b006b510 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5aed9d700 1 --2- 172.21.15.26:0/828988975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b006b510 0x7fa5b006e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0073480 tx=0x7fa5a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180ba7fb6a549b84 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5ad59a700 1 -- 172.21.15.26:0/828988975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800e070 con 0x7fa5b006b510 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5ad59a700 1 -- 172.21.15.26:0/828988975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800bb20 con 0x7fa5b006b510 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.263+0000 7fa5ad59a700 1 -- 172.21.15.26:0/828988975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800be90 con 0x7fa5b006b510 2024-01-16T14:54:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/828988975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b006b510 msgr2=0x7fa5b006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/828988975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b006b510 0x7fa5b006e370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0073480 tx=0x7fa5a800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/828988975 shutdown_connections 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/828988975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b006b510 0x7fa5b006e370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/828988975 >> v2:172.21.15.93:3300/0 conn(0x7fa5b00b57b0 0x7fa5b006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/828988975 >> 172.21.15.26:0/828988975 conn(0x7fa5b00d0aa0 msgr2=0x7fa5b00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/828988975 shutdown_connections 2024-01-16T14:54:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.264+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/828988975 wait complete. 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 Processor -- start 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 -- start start 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5b006b510 0x7fa5b01f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5b01478c0 0x7fa5b013f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00ae5a0 con 0x7fa5b00b57b0 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00ad990 con 0x7fa5b006b510 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5b4b8b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b00adc70 con 0x7fa5b01478c0 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.266+0000 7fa5add9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49488/0 (socket says 172.21.15.26:49488) 2024-01-16T14:54:50.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 -- 172.21.15.26:0/1563847932 learned_addr learned my addr 172.21.15.26:0/1563847932 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5ae59c700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5b006b510 0x7fa5b01f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5aed9d700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5b01478c0 0x7fa5b013f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 -- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5b01478c0 msgr2=0x7fa5b013f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5b01478c0 0x7fa5b013f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 -- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5b006b510 msgr2=0x7fa5b01f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5b006b510 0x7fa5b01f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 -- 172.21.15.26:0/1563847932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009a70 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5add9b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9b0 tx=0x7fa5a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4009710 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a400c850 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b00adf50 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400cbc0 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.267+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b01f7700 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.268+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b0075480 con 0x7fa5b00b57b0 2024-01-16T14:54:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.269+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa5a400cd20 con 0x7fa5b00b57b0 2024-01-16T14:54:50.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.269+0000 7fa59effd700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 0x7fa59006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.269+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5a4089bb0 con 0x7fa5b00b57b0 2024-01-16T14:54:50.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.269+0000 7fa5ae59c700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 0x7fa59006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.270+0000 7fa5ae59c700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 0x7fa59006d840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005910 tx=0x7fa5a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.274+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5a4054ab0 con 0x7fa5b00b57b0 2024-01-16T14:54:50.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.481+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5b01628a0 con 0x7fa59006b380 2024-01-16T14:54:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.489+0000 7fa59effd700 1 -- 172.21.15.26:0/1563847932 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264683 (secure 0 0 0) 0x7fa5a0001480 con 0x7fa59006b380 2024-01-16T14:54:50.475 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:50.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 msgr2=0x7fa59006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 0x7fa59006d840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005910 tx=0x7fa5a0009040 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 msgr2=0x7fa5b00aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9b0 tx=0x7fa5a400ed70 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 shutdown_connections 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5b01478c0 0x7fa5b013f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5b006b510 0x7fa5b01f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa59006b380 0x7fa59006d840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 --2- 172.21.15.26:0/1563847932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b00b57b0 0x7fa5b00aea30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 >> 172.21.15.26:0/1563847932 conn(0x7fa5b00d0aa0 msgr2=0x7fa5b0071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 shutdown_connections 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.493+0000 7fa5b4b8b700 1 -- 172.21.15.26:0/1563847932 wait complete. 2024-01-16T14:54:50.478 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:54:50.490 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-01-16T14:54:49.350728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2973696,"data_stored":1787728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.515941"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854111+0000","last_change":"2024-01-16T14:54:45.877716+0000","last_active":"2024-01-16T14:54:46.854111+0000","last_peered":"2024-01-16T14:54:46.854111+0000","last_clean":"2024-01-16T14:54:46.854111+0000","last_became_active":"2024-01-16T14:54:45.876462+0000","last_became_peered":"2024-01-16T14:54:45.876462+0000","last_unstale":"2024-01-16T14:54:46.854111+0000","last_undegraded":"2024-01-16T14:54:46.854111+0000","last_fullsized":"2024-01-16T14:54:46.854111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262923+0000","last_change":"2024-01-16T14:54:45.880623+0000","last_active":"2024-01-16T14:54:47.262923+0000","last_peered":"2024-01-16T14:54:47.262923+0000","last_clean":"2024-01-16T14:54:47.262923+0000","last_became_active":"2024-01-16T14:54:45.878584+0000","last_became_peered":"2024-01-16T14:54:45.878584+0000","last_unstale":"2024-01-16T14:54:47.262923+0000","last_undegraded":"2024-01-16T14:54:47.262923+0000","last_fullsized":"2024-01-16T14:54:47.262923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261939+0000","last_change":"2024-01-16T14:54:45.875065+0000","last_active":"2024-01-16T14:54:47.261939+0000","last_peered":"2024-01-16T14:54:47.261939+0000","last_clean":"2024-01-16T14:54:47.261939+0000","last_became_active":"2024-01-16T14:54:45.873809+0000","last_became_peered":"2024-01-16T14:54:45.873809+0000","last_unstale":"2024-01-16T14:54:47.261939+0000","last_undegraded":"2024-01-16T14:54:47.261939+0000","last_fullsized":"2024-01-16T14:54:47.261939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261991+0000","last_change":"2024-01-16T14:54:45.877233+0000","last_active":"2024-01-16T14:54:47.261991+0000","last_peered":"2024-01-16T14:54:47.261991+0000","last_clean":"2024-01-16T14:54:47.261991+0000","last_became_active":"2024-01-16T14:54:45.876940+0000","last_became_peered":"2024-01-16T14:54:45.876940+0000","last_unstale":"2024-01-16T14:54:47.261991+0000","last_undegraded":"2024-01-16T14:54:47.261991+0000","last_fullsized":"2024-01-16T14:54:47.261991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853429+0000","last_change":"2024-01-16T14:54:45.874709+0000","last_active":"2024-01-16T14:54:46.853429+0000","last_peered":"2024-01-16T14:54:46.853429+0000","last_clean":"2024-01-16T14:54:46.853429+0000","last_became_active":"2024-01-16T14:54:45.872900+0000","last_became_peered":"2024-01-16T14:54:45.872900+0000","last_unstale":"2024-01-16T14:54:46.853429+0000","last_undegraded":"2024-01-16T14:54:46.853429+0000","last_fullsized":"2024-01-16T14:54:46.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261215+0000","last_change":"2024-01-16T14:54:45.882475+0000","last_active":"2024-01-16T14:54:47.261215+0000","last_peered":"2024-01-16T14:54:47.261215+0000","last_clean":"2024-01-16T14:54:47.261215+0000","last_became_active":"2024-01-16T14:54:45.882150+0000","last_became_peered":"2024-01-16T14:54:45.882150+0000","last_unstale":"2024-01-16T14:54:47.261215+0000","last_undegraded":"2024-01-16T14:54:47.261215+0000","last_fullsized":"2024-01-16T14:54:47.261215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853589+0000","last_change":"2024-01-16T14:54:45.875750+0000","last_active":"2024-01-16T14:54:46.853589+0000","last_peered":"2024-01-16T14:54:46.853589+0000","last_clean":"2024-01-16T14:54:46.853589+0000","last_became_active":"2024-01-16T14:54:45.875474+0000","last_became_peered":"2024-01-16T14:54:45.875474+0000","last_unstale":"2024-01-16T14:54:46.853589+0000","last_undegraded":"2024-01-16T14:54:46.853589+0000","last_fullsized":"2024-01-16T14:54:46.853589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853586+0000","last_change":"2024-01-16T14:54:45.882976+0000","last_active":"2024-01-16T14:54:46.853586+0000","last_peered":"2024-01-16T14:54:46.853586+0000","last_clean":"2024-01-16T14:54:46.853586+0000","last_became_active":"2024-01-16T14:54:45.882628+0000","last_became_peered":"2024-01-16T14:54:45.882628+0000","last_unstale":"2024-01-16T14:54:46.853586+0000","last_undegraded":"2024-01-16T14:54:46.853586+0000","last_fullsized":"2024-01-16T14:54:46.853586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853730+0000","last_change":"2024-01-16T14:54:45.885967+0000","last_active":"2024-01-16T14:54:46.853730+0000","last_peered":"2024-01-16T14:54:46.853730+0000","last_clean":"2024-01-16T14:54:46.853730+0000","last_became_active":"2024-01-16T14:54:45.885700+0000","last_became_peered":"2024-01-16T14:54:45.885700+0000","last_unstale":"2024-01-16T14:54:46.853730+0000","last_undegraded":"2024-01-16T14:54:46.853730+0000","last_fullsized":"2024-01-16T14:54:46.853730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262320+0000","last_change":"2024-01-16T14:54:45.878748+0000","last_active":"2024-01-16T14:54:47.262320+0000","last_peered":"2024-01-16T14:54:47.262320+0000","last_clean":"2024-01-16T14:54:47.262320+0000","last_became_active":"2024-01-16T14:54:45.878446+0000","last_became_peered":"2024-01-16T14:54:45.878446+0000","last_unstale":"2024-01-16T14:54:47.262320+0000","last_undegraded":"2024-01-16T14:54:47.262320+0000","last_fullsized":"2024-01-16T14:54:47.262320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261357+0000","last_change":"2024-01-16T14:54:45.879232+0000","last_active":"2024-01-16T14:54:47.261357+0000","last_peered":"2024-01-16T14:54:47.261357+0000","last_clean":"2024-01-16T14:54:47.261357+0000","last_became_active":"2024-01-16T14:54:45.878875+0000","last_became_peered":"2024-01-16T14:54:45.878875+0000","last_unstale":"2024-01-16T14:54:47.261357+0000","last_undegraded":"2024-01-16T14:54:47.261357+0000","last_fullsized":"2024-01-16T14:54:47.261357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855341+0000","last_change":"2024-01-16T14:54:45.862373+0000","last_active":"2024-01-16T14:54:46.855341+0000","last_peered":"2024-01-16T14:54:46.855341+0000","last_clean":"2024-01-16T14:54:46.855341+0000","last_became_active":"2024-01-16T14:54:45.862110+0000","last_became_peered":"2024-01-16T14:54:45.862110+0000","last_unstale":"2024-01-16T14:54:46.855341+0000","last_undegraded":"2024-01-16T14:54:46.855341+0000","last_fullsized":"2024-01-16T14:54:46.855341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854063+0000","last_change":"2024-01-16T14:54:45.867064+0000","last_active":"2024-01-16T14:54:46.854063+0000","last_peered":"2024-01-16T14:54:46.854063+0000","last_clean":"2024-01-16T14:54:46.854063+0000","last_became_active":"2024-01-16T14:54:45.866257+0000","last_became_peered":"2024-01-16T14:54:45.866257+0000","last_unstale":"2024-01-16T14:54:46.854063+0000","last_undegraded":"2024-01-16T14:54:46.854063+0000","last_fullsized":"2024-01-16T14:54:46.854063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261852+0000","last_change":"2024-01-16T14:54:45.875851+0000","last_active":"2024-01-16T14:54:47.261852+0000","last_peered":"2024-01-16T14:54:47.261852+0000","last_clean":"2024-01-16T14:54:47.261852+0000","last_became_active":"2024-01-16T14:54:45.875202+0000","last_became_peered":"2024-01-16T14:54:45.875202+0000","last_unstale":"2024-01-16T14:54:47.261852+0000","last_undegraded":"2024-01-16T14:54:47.261852+0000","last_fullsized":"2024-01-16T14:54:47.261852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:35.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261833+0000","last_change":"2024-01-16T14:54:45.858130+0000","last_active":"2024-01-16T14:54:47.261833+0000","last_peered":"2024-01-16T14:54:47.261833+0000","last_clean":"2024-01-16T14:54:47.261833+0000","last_became_active":"2024-01-16T14:54:45.856812+0000","last_became_peered":"2024-01-16T14:54:45.856812+0000","last_unstale":"2024-01-16T14:54:47.261833+0000","last_undegraded":"2024-01-16T14:54:47.261833+0000","last_fullsized":"2024-01-16T14:54:47.261833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261530+0000","last_change":"2024-01-16T14:54:45.883516+0000","last_active":"2024-01-16T14:54:47.261530+0000","last_peered":"2024-01-16T14:54:47.261530+0000","last_clean":"2024-01-16T14:54:47.261530+0000","last_became_active":"2024-01-16T14:54:45.883259+0000","last_became_peered":"2024-01-16T14:54:45.883259+0000","last_unstale":"2024-01-16T14:54:47.261530+0000","last_undegraded":"2024-01-16T14:54:47.261530+0000","last_fullsized":"2024-01-16T14:54:47.261530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262193+0000","last_change":"2024-01-16T14:54:45.883796+0000","last_active":"2024-01-16T14:54:47.262193+0000","last_peered":"2024-01-16T14:54:47.262193+0000","last_clean":"2024-01-16T14:54:47.262193+0000","last_became_active":"2024-01-16T14:54:45.883558+0000","last_became_peered":"2024-01-16T14:54:45.883558+0000","last_unstale":"2024-01-16T14:54:47.262193+0000","last_undegraded":"2024-01-16T14:54:47.262193+0000","last_fullsized":"2024-01-16T14:54:47.262193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262894+0000","last_change":"2024-01-16T14:54:45.867763+0000","last_active":"2024-01-16T14:54:47.262894+0000","last_peered":"2024-01-16T14:54:47.262894+0000","last_clean":"2024-01-16T14:54:47.262894+0000","last_became_active":"2024-01-16T14:54:45.867441+0000","last_became_peered":"2024-01-16T14:54:45.867441+0000","last_unstale":"2024-01-16T14:54:47.262894+0000","last_undegraded":"2024-01-16T14:54:47.262894+0000","last_fullsized":"2024-01-16T14:54:47.262894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853971+0000","last_change":"2024-01-16T14:54:45.873151+0000","last_active":"2024-01-16T14:54:46.853971+0000","last_peered":"2024-01-16T14:54:46.853971+0000","last_clean":"2024-01-16T14:54:46.853971+0000","last_became_active":"2024-01-16T14:54:45.872841+0000","last_became_peered":"2024-01-16T14:54:45.872841+0000","last_unstale":"2024-01-16T14:54:46.853971+0000","last_undegraded":"2024-01-16T14:54:46.853971+0000","last_fullsized":"2024-01-16T14:54:46.853971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262224+0000","last_change":"2024-01-16T14:54:44.860003+0000","last_active":"2024-01-16T14:54:47.262224+0000","last_peered":"2024-01-16T14:54:47.262224+0000","last_clean":"2024-01-16T14:54:47.262224+0000","last_became_active":"2024-01-16T14:54:44.859774+0000","last_became_peered":"2024-01-16T14:54:44.859774+0000","last_unstale":"2024-01-16T14:54:47.262224+0000","last_undegraded":"2024-01-16T14:54:47.262224+0000","last_fullsized":"2024-01-16T14:54:47.262224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854240+0000","last_change":"2024-01-16T14:54:45.867265+0000","last_active":"2024-01-16T14:54:46.854240+0000","last_peered":"2024-01-16T14:54:46.854240+0000","last_clean":"2024-01-16T14:54:46.854240+0000","last_became_active":"2024-01-16T14:54:45.865756+0000","last_became_peered":"2024-01-16T14:54:45.865756+0000","last_unstale":"2024-01-16T14:54:46.854240+0000","last_undegraded":"2024-01-16T14:54:46.854240+0000","last_fullsized":"2024-01-16T14:54:46.854240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854088+0000","last_change":"2024-01-16T14:54:44.853662+0000","last_active":"2024-01-16T14:54:46.854088+0000","last_peered":"2024-01-16T14:54:46.854088+0000","last_clean":"2024-01-16T14:54:46.854088+0000","last_became_active":"2024-01-16T14:54:44.853379+0000","last_became_peered":"2024-01-16T14:54:44.853379+0000","last_unstale":"2024-01-16T14:54:46.854088+0000","last_undegraded":"2024-01-16T14:54:46.854088+0000","last_fullsized":"2024-01-16T14:54:46.854088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262160+0000","last_change":"2024-01-16T14:54:45.880017+0000","last_active":"2024-01-16T14:54:47.262160+0000","last_peered":"2024-01-16T14:54:47.262160+0000","last_clean":"2024-01-16T14:54:47.262160+0000","last_became_active":"2024-01-16T14:54:45.879369+0000","last_became_peered":"2024-01-16T14:54:45.879369+0000","last_unstale":"2024-01-16T14:54:47.262160+0000","last_undegraded":"2024-01-16T14:54:47.262160+0000","last_fullsized":"2024-01-16T14:54:47.262160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853963+0000","last_change":"2024-01-16T14:54:44.852663+0000","last_active":"2024-01-16T14:54:46.853963+0000","last_peered":"2024-01-16T14:54:46.853963+0000","last_clean":"2024-01-16T14:54:46.853963+0000","last_became_active":"2024-01-16T14:54:44.852346+0000","last_became_peered":"2024-01-16T14:54:44.852346+0000","last_unstale":"2024-01-16T14:54:46.853963+0000","last_undegraded":"2024-01-16T14:54:46.853963+0000","last_fullsized":"2024-01-16T14:54:46.853963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:54.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261811+0000","last_change":"2024-01-16T14:54:45.886559+0000","last_active":"2024-01-16T14:54:47.261811+0000","last_peered":"2024-01-16T14:54:47.261811+0000","last_clean":"2024-01-16T14:54:47.261811+0000","last_became_active":"2024-01-16T14:54:45.886343+0000","last_became_peered":"2024-01-16T14:54:45.886343+0000","last_unstale":"2024-01-16T14:54:47.261811+0000","last_undegraded":"2024-01-16T14:54:47.261811+0000","last_fullsized":"2024-01-16T14:54:47.261811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853331+0000","last_change":"2024-01-16T14:54:44.852311+0000","last_active":"2024-01-16T14:54:46.853331+0000","last_peered":"2024-01-16T14:54:46.853331+0000","last_clean":"2024-01-16T14:54:46.853331+0000","last_became_active":"2024-01-16T14:54:44.852018+0000","last_became_peered":"2024-01-16T14:54:44.852018+0000","last_unstale":"2024-01-16T14:54:46.853331+0000","last_undegraded":"2024-01-16T14:54:46.853331+0000","last_fullsized":"2024-01-16T14:54:46.853331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262110+0000","last_change":"2024-01-16T14:54:45.883384+0000","last_active":"2024-01-16T14:54:47.262110+0000","last_peered":"2024-01-16T14:54:47.262110+0000","last_clean":"2024-01-16T14:54:47.262110+0000","last_became_active":"2024-01-16T14:54:45.883094+0000","last_became_peered":"2024-01-16T14:54:45.883094+0000","last_unstale":"2024-01-16T14:54:47.262110+0000","last_undegraded":"2024-01-16T14:54:47.262110+0000","last_fullsized":"2024-01-16T14:54:47.262110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853863+0000","last_change":"2024-01-16T14:54:44.846655+0000","last_active":"2024-01-16T14:54:46.853863+0000","last_peered":"2024-01-16T14:54:46.853863+0000","last_clean":"2024-01-16T14:54:46.853863+0000","last_became_active":"2024-01-16T14:54:44.846359+0000","last_became_peered":"2024-01-16T14:54:44.846359+0000","last_unstale":"2024-01-16T14:54:46.853863+0000","last_undegraded":"2024-01-16T14:54:46.853863+0000","last_fullsized":"2024-01-16T14:54:46.853863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:28.971339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854823+0000","last_change":"2024-01-16T14:54:45.857956+0000","last_active":"2024-01-16T14:54:46.854823+0000","last_peered":"2024-01-16T14:54:46.854823+0000","last_clean":"2024-01-16T14:54:46.854823+0000","last_became_active":"2024-01-16T14:54:45.857645+0000","last_became_peered":"2024-01-16T14:54:45.857645+0000","last_unstale":"2024-01-16T14:54:46.854823+0000","last_undegraded":"2024-01-16T14:54:46.854823+0000","last_fullsized":"2024-01-16T14:54:46.854823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:39.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261584+0000","last_change":"2024-01-16T14:54:44.851910+0000","last_active":"2024-01-16T14:54:47.261584+0000","last_peered":"2024-01-16T14:54:47.261584+0000","last_clean":"2024-01-16T14:54:47.261584+0000","last_became_active":"2024-01-16T14:54:44.851631+0000","last_became_peered":"2024-01-16T14:54:44.851631+0000","last_unstale":"2024-01-16T14:54:47.261584+0000","last_undegraded":"2024-01-16T14:54:47.261584+0000","last_fullsized":"2024-01-16T14:54:47.261584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:17.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262365+0000","last_change":"2024-01-16T14:54:45.875708+0000","last_active":"2024-01-16T14:54:47.262365+0000","last_peered":"2024-01-16T14:54:47.262365+0000","last_clean":"2024-01-16T14:54:47.262365+0000","last_became_active":"2024-01-16T14:54:45.875328+0000","last_became_peered":"2024-01-16T14:54:45.875328+0000","last_unstale":"2024-01-16T14:54:47.262365+0000","last_undegraded":"2024-01-16T14:54:47.262365+0000","last_fullsized":"2024-01-16T14:54:47.262365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853648+0000","last_change":"2024-01-16T14:54:44.855286+0000","last_active":"2024-01-16T14:54:46.853648+0000","last_peered":"2024-01-16T14:54:46.853648+0000","last_clean":"2024-01-16T14:54:46.853648+0000","last_became_active":"2024-01-16T14:54:44.854465+0000","last_became_peered":"2024-01-16T14:54:44.854465+0000","last_unstale":"2024-01-16T14:54:46.853648+0000","last_undegraded":"2024-01-16T14:54:46.853648+0000","last_fullsized":"2024-01-16T14:54:46.853648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853759+0000","last_change":"2024-01-16T14:54:45.868676+0000","last_active":"2024-01-16T14:54:46.853759+0000","last_peered":"2024-01-16T14:54:46.853759+0000","last_clean":"2024-01-16T14:54:46.853759+0000","last_became_active":"2024-01-16T14:54:45.868354+0000","last_became_peered":"2024-01-16T14:54:45.868354+0000","last_unstale":"2024-01-16T14:54:46.853759+0000","last_undegraded":"2024-01-16T14:54:46.853759+0000","last_fullsized":"2024-01-16T14:54:46.853759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261155+0000","last_change":"2024-01-16T14:54:44.855323+0000","last_active":"2024-01-16T14:54:47.261155+0000","last_peered":"2024-01-16T14:54:47.261155+0000","last_clean":"2024-01-16T14:54:47.261155+0000","last_became_active":"2024-01-16T14:54:44.855035+0000","last_became_peered":"2024-01-16T14:54:44.855035+0000","last_unstale":"2024-01-16T14:54:47.261155+0000","last_undegraded":"2024-01-16T14:54:47.261155+0000","last_fullsized":"2024-01-16T14:54:47.261155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:33.068253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262370+0000","last_change":"2024-01-16T14:54:45.864396+0000","last_active":"2024-01-16T14:54:47.262370+0000","last_peered":"2024-01-16T14:54:47.262370+0000","last_clean":"2024-01-16T14:54:47.262370+0000","last_became_active":"2024-01-16T14:54:45.864052+0000","last_became_peered":"2024-01-16T14:54:45.864052+0000","last_unstale":"2024-01-16T14:54:47.262370+0000","last_undegraded":"2024-01-16T14:54:47.262370+0000","last_fullsized":"2024-01-16T14:54:47.262370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262332+0000","last_change":"2024-01-16T14:54:44.857192+0000","last_active":"2024-01-16T14:54:47.262332+0000","last_peered":"2024-01-16T14:54:47.262332+0000","last_clean":"2024-01-16T14:54:47.262332+0000","last_became_active":"2024-01-16T14:54:44.856880+0000","last_became_peered":"2024-01-16T14:54:44.856880+0000","last_unstale":"2024-01-16T14:54:47.262332+0000","last_undegraded":"2024-01-16T14:54:47.262332+0000","last_fullsized":"2024-01-16T14:54:47.262332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261868+0000","last_change":"2024-01-16T14:54:45.884208+0000","last_active":"2024-01-16T14:54:47.261868+0000","last_peered":"2024-01-16T14:54:47.261868+0000","last_clean":"2024-01-16T14:54:47.261868+0000","last_became_active":"2024-01-16T14:54:45.882540+0000","last_became_peered":"2024-01-16T14:54:45.882540+0000","last_unstale":"2024-01-16T14:54:47.261868+0000","last_undegraded":"2024-01-16T14:54:47.261868+0000","last_fullsized":"2024-01-16T14:54:47.261868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261721+0000","last_change":"2024-01-16T14:54:44.854856+0000","last_active":"2024-01-16T14:54:47.261721+0000","last_peered":"2024-01-16T14:54:47.261721+0000","last_clean":"2024-01-16T14:54:47.261721+0000","last_became_active":"2024-01-16T14:54:44.854543+0000","last_became_peered":"2024-01-16T14:54:44.854543+0000","last_unstale":"2024-01-16T14:54:47.261721+0000","last_undegraded":"2024-01-16T14:54:47.261721+0000","last_fullsized":"2024-01-16T14:54:47.261721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262312+0000","last_change":"2024-01-16T14:54:45.880240+0000","last_active":"2024-01-16T14:54:47.262312+0000","last_peered":"2024-01-16T14:54:47.262312+0000","last_clean":"2024-01-16T14:54:47.262312+0000","last_became_active":"2024-01-16T14:54:45.879764+0000","last_became_peered":"2024-01-16T14:54:45.879764+0000","last_unstale":"2024-01-16T14:54:47.262312+0000","last_undegraded":"2024-01-16T14:54:47.262312+0000","last_fullsized":"2024-01-16T14:54:47.262312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262763+0000","last_change":"2024-01-16T14:54:44.855449+0000","last_active":"2024-01-16T14:54:47.262763+0000","last_peered":"2024-01-16T14:54:47.262763+0000","last_clean":"2024-01-16T14:54:47.262763+0000","last_became_active":"2024-01-16T14:54:44.855037+0000","last_became_peered":"2024-01-16T14:54:44.855037+0000","last_unstale":"2024-01-16T14:54:47.262763+0000","last_undegraded":"2024-01-16T14:54:47.262763+0000","last_fullsized":"2024-01-16T14:54:47.262763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855392+0000","last_change":"2024-01-16T14:54:45.870932+0000","last_active":"2024-01-16T14:54:46.855392+0000","last_peered":"2024-01-16T14:54:46.855392+0000","last_clean":"2024-01-16T14:54:46.855392+0000","last_became_active":"2024-01-16T14:54:45.870016+0000","last_became_peered":"2024-01-16T14:54:45.870016+0000","last_unstale":"2024-01-16T14:54:46.855392+0000","last_undegraded":"2024-01-16T14:54:46.855392+0000","last_fullsized":"2024-01-16T14:54:46.855392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261353+0000","last_change":"2024-01-16T14:54:44.854762+0000","last_active":"2024-01-16T14:54:47.261353+0000","last_peered":"2024-01-16T14:54:47.261353+0000","last_clean":"2024-01-16T14:54:47.261353+0000","last_became_active":"2024-01-16T14:54:44.854507+0000","last_became_peered":"2024-01-16T14:54:44.854507+0000","last_unstale":"2024-01-16T14:54:47.261353+0000","last_undegraded":"2024-01-16T14:54:47.261353+0000","last_fullsized":"2024-01-16T14:54:47.261353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853580+0000","last_change":"2024-01-16T14:54:45.879814+0000","last_active":"2024-01-16T14:54:46.853580+0000","last_peered":"2024-01-16T14:54:46.853580+0000","last_clean":"2024-01-16T14:54:46.853580+0000","last_became_active":"2024-01-16T14:54:45.873897+0000","last_became_peered":"2024-01-16T14:54:45.873897+0000","last_unstale":"2024-01-16T14:54:46.853580+0000","last_undegraded":"2024-01-16T14:54:46.853580+0000","last_fullsized":"2024-01-16T14:54:46.853580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262205+0000","last_change":"2024-01-16T14:54:44.860218+0000","last_active":"2024-01-16T14:54:47.262205+0000","last_peered":"2024-01-16T14:54:47.262205+0000","last_clean":"2024-01-16T14:54:47.262205+0000","last_became_active":"2024-01-16T14:54:44.859920+0000","last_became_peered":"2024-01-16T14:54:44.859920+0000","last_unstale":"2024-01-16T14:54:47.262205+0000","last_undegraded":"2024-01-16T14:54:47.262205+0000","last_fullsized":"2024-01-16T14:54:47.262205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262345+0000","last_change":"2024-01-16T14:54:45.875405+0000","last_active":"2024-01-16T14:54:47.262345+0000","last_peered":"2024-01-16T14:54:47.262345+0000","last_clean":"2024-01-16T14:54:47.262345+0000","last_became_active":"2024-01-16T14:54:45.874428+0000","last_became_peered":"2024-01-16T14:54:45.874428+0000","last_unstale":"2024-01-16T14:54:47.262345+0000","last_undegraded":"2024-01-16T14:54:47.262345+0000","last_fullsized":"2024-01-16T14:54:47.262345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:27.912278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262076+0000","last_change":"2024-01-16T14:54:44.857601+0000","last_active":"2024-01-16T14:54:47.262076+0000","last_peered":"2024-01-16T14:54:47.262076+0000","last_clean":"2024-01-16T14:54:47.262076+0000","last_became_active":"2024-01-16T14:54:44.857282+0000","last_became_peered":"2024-01-16T14:54:44.857282+0000","last_unstale":"2024-01-16T14:54:47.262076+0000","last_undegraded":"2024-01-16T14:54:47.262076+0000","last_fullsized":"2024-01-16T14:54:47.262076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853602+0000","last_change":"2024-01-16T14:54:45.873626+0000","last_active":"2024-01-16T14:54:46.853602+0000","last_peered":"2024-01-16T14:54:46.853602+0000","last_clean":"2024-01-16T14:54:46.853602+0000","last_became_active":"2024-01-16T14:54:45.872632+0000","last_became_peered":"2024-01-16T14:54:45.872632+0000","last_unstale":"2024-01-16T14:54:46.853602+0000","last_undegraded":"2024-01-16T14:54:46.853602+0000","last_fullsized":"2024-01-16T14:54:46.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261375+0000","last_change":"2024-01-16T14:54:44.852946+0000","last_active":"2024-01-16T14:54:47.261375+0000","last_peered":"2024-01-16T14:54:47.261375+0000","last_clean":"2024-01-16T14:54:47.261375+0000","last_became_active":"2024-01-16T14:54:44.852682+0000","last_became_peered":"2024-01-16T14:54:44.852682+0000","last_unstale":"2024-01-16T14:54:47.261375+0000","last_undegraded":"2024-01-16T14:54:47.261375+0000","last_fullsized":"2024-01-16T14:54:47.261375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261947+0000","last_change":"2024-01-16T14:54:45.877152+0000","last_active":"2024-01-16T14:54:47.261947+0000","last_peered":"2024-01-16T14:54:47.261947+0000","last_clean":"2024-01-16T14:54:47.261947+0000","last_became_active":"2024-01-16T14:54:45.876716+0000","last_became_peered":"2024-01-16T14:54:45.876716+0000","last_unstale":"2024-01-16T14:54:47.261947+0000","last_undegraded":"2024-01-16T14:54:47.261947+0000","last_fullsized":"2024-01-16T14:54:47.261947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261401+0000","last_change":"2024-01-16T14:54:44.857725+0000","last_active":"2024-01-16T14:54:47.261401+0000","last_peered":"2024-01-16T14:54:47.261401+0000","last_clean":"2024-01-16T14:54:47.261401+0000","last_became_active":"2024-01-16T14:54:44.857273+0000","last_became_peered":"2024-01-16T14:54:44.857273+0000","last_unstale":"2024-01-16T14:54:47.261401+0000","last_undegraded":"2024-01-16T14:54:47.261401+0000","last_fullsized":"2024-01-16T14:54:47.261401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853385+0000","last_change":"2024-01-16T14:54:45.868095+0000","last_active":"2024-01-16T14:54:46.853385+0000","last_peered":"2024-01-16T14:54:46.853385+0000","last_clean":"2024-01-16T14:54:46.853385+0000","last_became_active":"2024-01-16T14:54:45.867008+0000","last_became_peered":"2024-01-16T14:54:45.867008+0000","last_unstale":"2024-01-16T14:54:46.853385+0000","last_undegraded":"2024-01-16T14:54:46.853385+0000","last_fullsized":"2024-01-16T14:54:46.853385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855353+0000","last_change":"2024-01-16T14:54:44.853674+0000","last_active":"2024-01-16T14:54:46.855353+0000","last_peered":"2024-01-16T14:54:46.855353+0000","last_clean":"2024-01-16T14:54:46.855353+0000","last_became_active":"2024-01-16T14:54:44.853079+0000","last_became_peered":"2024-01-16T14:54:44.853079+0000","last_unstale":"2024-01-16T14:54:46.855353+0000","last_undegraded":"2024-01-16T14:54:46.855353+0000","last_fullsized":"2024-01-16T14:54:46.855353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:31.747933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853861+0000","last_change":"2024-01-16T14:54:45.875678+0000","last_active":"2024-01-16T14:54:46.853861+0000","last_peered":"2024-01-16T14:54:46.853861+0000","last_clean":"2024-01-16T14:54:46.853861+0000","last_became_active":"2024-01-16T14:54:45.875400+0000","last_became_peered":"2024-01-16T14:54:45.875400+0000","last_unstale":"2024-01-16T14:54:46.853861+0000","last_undegraded":"2024-01-16T14:54:46.853861+0000","last_fullsized":"2024-01-16T14:54:46.853861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262644+0000","last_change":"2024-01-16T14:54:44.858592+0000","last_active":"2024-01-16T14:54:47.262644+0000","last_peered":"2024-01-16T14:54:47.262644+0000","last_clean":"2024-01-16T14:54:47.262644+0000","last_became_active":"2024-01-16T14:54:44.858275+0000","last_became_peered":"2024-01-16T14:54:44.858275+0000","last_unstale":"2024-01-16T14:54:47.262644+0000","last_undegraded":"2024-01-16T14:54:47.262644+0000","last_fullsized":"2024-01-16T14:54:47.262644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262293+0000","last_change":"2024-01-16T14:54:45.877869+0000","last_active":"2024-01-16T14:54:47.262293+0000","last_peered":"2024-01-16T14:54:47.262293+0000","last_clean":"2024-01-16T14:54:47.262293+0000","last_became_active":"2024-01-16T14:54:45.876905+0000","last_became_peered":"2024-01-16T14:54:45.876905+0000","last_unstale":"2024-01-16T14:54:47.262293+0000","last_undegraded":"2024-01-16T14:54:47.262293+0000","last_fullsized":"2024-01-16T14:54:47.262293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:47:51.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261976+0000","last_change":"2024-01-16T14:54:44.855986+0000","last_active":"2024-01-16T14:54:47.261976+0000","last_peered":"2024-01-16T14:54:47.261976+0000","last_clean":"2024-01-16T14:54:47.261976+0000","last_became_active":"2024-01-16T14:54:44.855690+0000","last_became_peered":"2024-01-16T14:54:44.855690+0000","last_unstale":"2024-01-16T14:54:47.261976+0000","last_undegraded":"2024-01-16T14:54:47.261976+0000","last_fullsized":"2024-01-16T14:54:47.261976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855013+0000","last_change":"2024-01-16T14:54:45.878441+0000","last_active":"2024-01-16T14:54:46.855013+0000","last_peered":"2024-01-16T14:54:46.855013+0000","last_clean":"2024-01-16T14:54:46.855013+0000","last_became_active":"2024-01-16T14:54:45.878011+0000","last_became_peered":"2024-01-16T14:54:45.878011+0000","last_unstale":"2024-01-16T14:54:46.855013+0000","last_undegraded":"2024-01-16T14:54:46.855013+0000","last_fullsized":"2024-01-16T14:54:46.855013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:05.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261593+0000","last_change":"2024-01-16T14:54:45.869056+0000","last_active":"2024-01-16T14:54:47.261593+0000","last_peered":"2024-01-16T14:54:47.261593+0000","last_clean":"2024-01-16T14:54:47.261593+0000","last_became_active":"2024-01-16T14:54:45.868748+0000","last_became_peered":"2024-01-16T14:54:45.868748+0000","last_unstale":"2024-01-16T14:54:47.261593+0000","last_undegraded":"2024-01-16T14:54:47.261593+0000","last_fullsized":"2024-01-16T14:54:47.261593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261832+0000","last_change":"2024-01-16T14:54:44.853961+0000","last_active":"2024-01-16T14:54:47.261832+0000","last_peered":"2024-01-16T14:54:47.261832+0000","last_clean":"2024-01-16T14:54:47.261832+0000","last_became_active":"2024-01-16T14:54:44.853610+0000","last_became_peered":"2024-01-16T14:54:44.853610+0000","last_unstale":"2024-01-16T14:54:47.261832+0000","last_undegraded":"2024-01-16T14:54:47.261832+0000","last_fullsized":"2024-01-16T14:54:47.261832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853929+0000","last_change":"2024-01-16T14:54:45.872254+0000","last_active":"2024-01-16T14:54:46.853929+0000","last_peered":"2024-01-16T14:54:46.853929+0000","last_clean":"2024-01-16T14:54:46.853929+0000","last_became_active":"2024-01-16T14:54:45.870597+0000","last_became_peered":"2024-01-16T14:54:45.870597+0000","last_unstale":"2024-01-16T14:54:46.853929+0000","last_undegraded":"2024-01-16T14:54:46.853929+0000","last_fullsized":"2024-01-16T14:54:46.853929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261437+0000","last_change":"2024-01-16T14:54:44.854781+0000","last_active":"2024-01-16T14:54:47.261437+0000","last_peered":"2024-01-16T14:54:47.261437+0000","last_clean":"2024-01-16T14:54:47.261437+0000","last_became_active":"2024-01-16T14:54:44.854461+0000","last_became_peered":"2024-01-16T14:54:44.854461+0000","last_unstale":"2024-01-16T14:54:47.261437+0000","last_undegraded":"2024-01-16T14:54:47.261437+0000","last_fullsized":"2024-01-16T14:54:47.261437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T14:54:45.859524+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:54:45.859524+0000","last_peered":"2024-01-16T14:54:45.859524+0000","last_clean":"2024-01-16T14:54:45.859524+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:54:45.859524+0000","last_undegraded":"2024-01-16T14:54:45.859524+0000","last_fullsized":"2024-01-16T14:54:45.859524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261727+0000","last_change":"2024-01-16T14:54:45.875199+0000","last_active":"2024-01-16T14:54:47.261727+0000","last_peered":"2024-01-16T14:54:47.261727+0000","last_clean":"2024-01-16T14:54:47.261727+0000","last_became_active":"2024-01-16T14:54:45.874049+0000","last_became_peered":"2024-01-16T14:54:45.874049+0000","last_unstale":"2024-01-16T14:54:47.261727+0000","last_undegraded":"2024-01-16T14:54:47.261727+0000","last_fullsized":"2024-01-16T14:54:47.261727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:29.976614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853314+0000","last_change":"2024-01-16T14:54:44.857691+0000","last_active":"2024-01-16T14:54:46.853314+0000","last_peered":"2024-01-16T14:54:46.853314+0000","last_clean":"2024-01-16T14:54:46.853314+0000","last_became_active":"2024-01-16T14:54:44.857407+0000","last_became_peered":"2024-01-16T14:54:44.857407+0000","last_unstale":"2024-01-16T14:54:46.853314+0000","last_undegraded":"2024-01-16T14:54:46.853314+0000","last_fullsized":"2024-01-16T14:54:46.853314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853856+0000","last_change":"2024-01-16T14:54:44.853542+0000","last_active":"2024-01-16T14:54:46.853856+0000","last_peered":"2024-01-16T14:54:46.853856+0000","last_clean":"2024-01-16T14:54:46.853856+0000","last_became_active":"2024-01-16T14:54:44.853273+0000","last_became_peered":"2024-01-16T14:54:44.853273+0000","last_unstale":"2024-01-16T14:54:46.853856+0000","last_undegraded":"2024-01-16T14:54:46.853856+0000","last_fullsized":"2024-01-16T14:54:46.853856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:56.322689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262718+0000","last_change":"2024-01-16T14:54:45.868977+0000","last_active":"2024-01-16T14:54:47.262718+0000","last_peered":"2024-01-16T14:54:47.262718+0000","last_clean":"2024-01-16T14:54:47.262718+0000","last_became_active":"2024-01-16T14:54:45.868675+0000","last_became_peered":"2024-01-16T14:54:45.868675+0000","last_unstale":"2024-01-16T14:54:47.262718+0000","last_undegraded":"2024-01-16T14:54:47.262718+0000","last_fullsized":"2024-01-16T14:54:47.262718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262724+0000","last_change":"2024-01-16T14:54:44.855466+0000","last_active":"2024-01-16T14:54:47.262724+0000","last_peered":"2024-01-16T14:54:47.262724+0000","last_clean":"2024-01-16T14:54:47.262724+0000","last_became_active":"2024-01-16T14:54:44.855144+0000","last_became_peered":"2024-01-16T14:54:44.855144+0000","last_unstale":"2024-01-16T14:54:47.262724+0000","last_undegraded":"2024-01-16T14:54:47.262724+0000","last_fullsized":"2024-01-16T14:54:47.262724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854544+0000","last_change":"2024-01-16T14:54:45.875954+0000","last_active":"2024-01-16T14:54:46.854544+0000","last_peered":"2024-01-16T14:54:46.854544+0000","last_clean":"2024-01-16T14:54:46.854544+0000","last_became_active":"2024-01-16T14:54:45.875635+0000","last_became_peered":"2024-01-16T14:54:45.875635+0000","last_unstale":"2024-01-16T14:54:46.854544+0000","last_undegraded":"2024-01-16T14:54:46.854544+0000","last_fullsized":"2024-01-16T14:54:46.854544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262344+0000","last_change":"2024-01-16T14:54:44.858826+0000","last_active":"2024-01-16T14:54:47.262344+0000","last_peered":"2024-01-16T14:54:47.262344+0000","last_clean":"2024-01-16T14:54:47.262344+0000","last_became_active":"2024-01-16T14:54:44.858528+0000","last_became_peered":"2024-01-16T14:54:44.858528+0000","last_unstale":"2024-01-16T14:54:47.262344+0000","last_undegraded":"2024-01-16T14:54:47.262344+0000","last_fullsized":"2024-01-16T14:54:47.262344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262074+0000","last_change":"2024-01-16T14:54:45.872205+0000","last_active":"2024-01-16T14:54:47.262074+0000","last_peered":"2024-01-16T14:54:47.262074+0000","last_clean":"2024-01-16T14:54:47.262074+0000","last_became_active":"2024-01-16T14:54:45.871893+0000","last_became_peered":"2024-01-16T14:54:45.871893+0000","last_unstale":"2024-01-16T14:54:47.262074+0000","last_undegraded":"2024-01-16T14:54:47.262074+0000","last_fullsized":"2024-01-16T14:54:47.262074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853147+0000","last_change":"2024-01-16T14:54:44.853929+0000","last_active":"2024-01-16T14:54:46.853147+0000","last_peered":"2024-01-16T14:54:46.853147+0000","last_clean":"2024-01-16T14:54:46.853147+0000","last_became_active":"2024-01-16T14:54:44.853655+0000","last_became_peered":"2024-01-16T14:54:44.853655+0000","last_unstale":"2024-01-16T14:54:46.853147+0000","last_undegraded":"2024-01-16T14:54:46.853147+0000","last_fullsized":"2024-01-16T14:54:46.853147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855031+0000","last_change":"2024-01-16T14:54:45.870957+0000","last_active":"2024-01-16T14:54:46.855031+0000","last_peered":"2024-01-16T14:54:46.855031+0000","last_clean":"2024-01-16T14:54:46.855031+0000","last_became_active":"2024-01-16T14:54:45.870672+0000","last_became_peered":"2024-01-16T14:54:45.870672+0000","last_unstale":"2024-01-16T14:54:46.855031+0000","last_undegraded":"2024-01-16T14:54:46.855031+0000","last_fullsized":"2024-01-16T14:54:46.855031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:19.153610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261884+0000","last_change":"2024-01-16T14:54:44.859174+0000","last_active":"2024-01-16T14:54:47.261884+0000","last_peered":"2024-01-16T14:54:47.261884+0000","last_clean":"2024-01-16T14:54:47.261884+0000","last_became_active":"2024-01-16T14:54:44.858845+0000","last_became_peered":"2024-01-16T14:54:44.858845+0000","last_unstale":"2024-01-16T14:54:47.261884+0000","last_undegraded":"2024-01-16T14:54:47.261884+0000","last_fullsized":"2024-01-16T14:54:47.261884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853729+0000","last_change":"2024-01-16T14:54:45.865608+0000","last_active":"2024-01-16T14:54:46.853729+0000","last_peered":"2024-01-16T14:54:46.853729+0000","last_clean":"2024-01-16T14:54:46.853729+0000","last_became_active":"2024-01-16T14:54:45.864856+0000","last_became_peered":"2024-01-16T14:54:45.864856+0000","last_unstale":"2024-01-16T14:54:46.853729+0000","last_undegraded":"2024-01-16T14:54:46.853729+0000","last_fullsized":"2024-01-16T14:54:46.853729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262559+0000","last_change":"2024-01-16T14:54:44.857846+0000","last_active":"2024-01-16T14:54:47.262559+0000","last_peered":"2024-01-16T14:54:47.262559+0000","last_clean":"2024-01-16T14:54:47.262559+0000","last_became_active":"2024-01-16T14:54:44.857244+0000","last_became_peered":"2024-01-16T14:54:44.857244+0000","last_unstale":"2024-01-16T14:54:47.262559+0000","last_undegraded":"2024-01-16T14:54:47.262559+0000","last_fullsized":"2024-01-16T14:54:47.262559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854055+0000","last_change":"2024-01-16T14:54:45.877240+0000","last_active":"2024-01-16T14:54:46.854055+0000","last_peered":"2024-01-16T14:54:46.854055+0000","last_clean":"2024-01-16T14:54:46.854055+0000","last_became_active":"2024-01-16T14:54:45.876921+0000","last_became_peered":"2024-01-16T14:54:45.876921+0000","last_unstale":"2024-01-16T14:54:46.854055+0000","last_undegraded":"2024-01-16T14:54:46.854055+0000","last_fullsized":"2024-01-16T14:54:46.854055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853720+0000","last_change":"2024-01-16T14:54:44.841886+0000","last_active":"2024-01-16T14:54:46.853720+0000","last_peered":"2024-01-16T14:54:46.853720+0000","last_clean":"2024-01-16T14:54:46.853720+0000","last_became_active":"2024-01-16T14:54:44.841590+0000","last_became_peered":"2024-01-16T14:54:44.841590+0000","last_unstale":"2024-01-16T14:54:46.853720+0000","last_undegraded":"2024-01-16T14:54:46.853720+0000","last_fullsized":"2024-01-16T14:54:46.853720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261806+0000","last_change":"2024-01-16T14:54:45.861293+0000","last_active":"2024-01-16T14:54:47.261806+0000","last_peered":"2024-01-16T14:54:47.261806+0000","last_clean":"2024-01-16T14:54:47.261806+0000","last_became_active":"2024-01-16T14:54:45.860966+0000","last_became_peered":"2024-01-16T14:54:45.860966+0000","last_unstale":"2024-01-16T14:54:47.261806+0000","last_undegraded":"2024-01-16T14:54:47.261806+0000","last_fullsized":"2024-01-16T14:54:47.261806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262495+0000","last_change":"2024-01-16T14:54:44.859596+0000","last_active":"2024-01-16T14:54:47.262495+0000","last_peered":"2024-01-16T14:54:47.262495+0000","last_clean":"2024-01-16T14:54:47.262495+0000","last_became_active":"2024-01-16T14:54:44.859150+0000","last_became_peered":"2024-01-16T14:54:44.859150+0000","last_unstale":"2024-01-16T14:54:47.262495+0000","last_undegraded":"2024-01-16T14:54:47.262495+0000","last_fullsized":"2024-01-16T14:54:47.262495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853762+0000","last_change":"2024-01-16T14:54:45.884933+0000","last_active":"2024-01-16T14:54:46.853762+0000","last_peered":"2024-01-16T14:54:46.853762+0000","last_clean":"2024-01-16T14:54:46.853762+0000","last_became_active":"2024-01-16T14:54:45.884688+0000","last_became_peered":"2024-01-16T14:54:45.884688+0000","last_unstale":"2024-01-16T14:54:46.853762+0000","last_undegraded":"2024-01-16T14:54:46.853762+0000","last_fullsized":"2024-01-16T14:54:46.853762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262635+0000","last_change":"2024-01-16T14:54:44.852405+0000","last_active":"2024-01-16T14:54:47.262635+0000","last_peered":"2024-01-16T14:54:47.262635+0000","last_clean":"2024-01-16T14:54:47.262635+0000","last_became_active":"2024-01-16T14:54:44.852011+0000","last_became_peered":"2024-01-16T14:54:44.852011+0000","last_unstale":"2024-01-16T14:54:47.262635+0000","last_undegraded":"2024-01-16T14:54:47.262635+0000","last_fullsized":"2024-01-16T14:54:47.262635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261800+0000","last_change":"2024-01-16T14:54:45.883630+0000","last_active":"2024-01-16T14:54:47.261800+0000","last_peered":"2024-01-16T14:54:47.261800+0000","last_clean":"2024-01-16T14:54:47.261800+0000","last_became_active":"2024-01-16T14:54:45.882901+0000","last_became_peered":"2024-01-16T14:54:45.882901+0000","last_unstale":"2024-01-16T14:54:47.261800+0000","last_undegraded":"2024-01-16T14:54:47.261800+0000","last_fullsized":"2024-01-16T14:54:47.261800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262954+0000","last_change":"2024-01-16T14:54:44.853825+0000","last_active":"2024-01-16T14:54:47.262954+0000","last_peered":"2024-01-16T14:54:47.262954+0000","last_clean":"2024-01-16T14:54:47.262954+0000","last_became_active":"2024-01-16T14:54:44.853509+0000","last_became_peered":"2024-01-16T14:54:44.853509+0000","last_unstale":"2024-01-16T14:54:47.262954+0000","last_undegraded":"2024-01-16T14:54:47.262954+0000","last_fullsized":"2024-01-16T14:54:47.262954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853960+0000","last_change":"2024-01-16T14:54:45.878029+0000","last_active":"2024-01-16T14:54:46.853960+0000","last_peered":"2024-01-16T14:54:46.853960+0000","last_clean":"2024-01-16T14:54:46.853960+0000","last_became_active":"2024-01-16T14:54:45.877266+0000","last_became_peered":"2024-01-16T14:54:45.877266+0000","last_unstale":"2024-01-16T14:54:46.853960+0000","last_undegraded":"2024-01-16T14:54:46.853960+0000","last_fullsized":"2024-01-16T14:54:46.853960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261228+0000","last_change":"2024-01-16T14:54:45.876962+0000","last_active":"2024-01-16T14:54:47.261228+0000","last_peered":"2024-01-16T14:54:47.261228+0000","last_clean":"2024-01-16T14:54:47.261228+0000","last_became_active":"2024-01-16T14:54:45.876358+0000","last_became_peered":"2024-01-16T14:54:45.876358+0000","last_unstale":"2024-01-16T14:54:47.261228+0000","last_undegraded":"2024-01-16T14:54:47.261228+0000","last_fullsized":"2024-01-16T14:54:47.261228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261492+0000","last_change":"2024-01-16T14:54:45.874788+0000","last_active":"2024-01-16T14:54:47.261492+0000","last_peered":"2024-01-16T14:54:47.261492+0000","last_clean":"2024-01-16T14:54:47.261492+0000","last_became_active":"2024-01-16T14:54:45.874453+0000","last_became_peered":"2024-01-16T14:54:45.874453+0000","last_unstale":"2024-01-16T14:54:47.261492+0000","last_undegraded":"2024-01-16T14:54:47.261492+0000","last_fullsized":"2024-01-16T14:54:47.261492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261754+0000","last_change":"2024-01-16T14:54:45.877374+0000","last_active":"2024-01-16T14:54:47.261754+0000","last_peered":"2024-01-16T14:54:47.261754+0000","last_clean":"2024-01-16T14:54:47.261754+0000","last_became_active":"2024-01-16T14:54:45.877118+0000","last_became_peered":"2024-01-16T14:54:45.877118+0000","last_unstale":"2024-01-16T14:54:47.261754+0000","last_undegraded":"2024-01-16T14:54:47.261754+0000","last_fullsized":"2024-01-16T14:54:47.261754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262220+0000","last_change":"2024-01-16T14:54:45.875476+0000","last_active":"2024-01-16T14:54:47.262220+0000","last_peered":"2024-01-16T14:54:47.262220+0000","last_clean":"2024-01-16T14:54:47.262220+0000","last_became_active":"2024-01-16T14:54:45.874303+0000","last_became_peered":"2024-01-16T14:54:45.874303+0000","last_unstale":"2024-01-16T14:54:47.262220+0000","last_undegraded":"2024-01-16T14:54:47.262220+0000","last_fullsized":"2024-01-16T14:54:47.262220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854023+0000","last_change":"2024-01-16T14:54:45.866307+0000","last_active":"2024-01-16T14:54:46.854023+0000","last_peered":"2024-01-16T14:54:46.854023+0000","last_clean":"2024-01-16T14:54:46.854023+0000","last_became_active":"2024-01-16T14:54:45.865419+0000","last_became_peered":"2024-01-16T14:54:45.865419+0000","last_unstale":"2024-01-16T14:54:46.854023+0000","last_undegraded":"2024-01-16T14:54:46.854023+0000","last_fullsized":"2024-01-16T14:54:46.854023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262723+0000","last_change":"2024-01-16T14:54:45.866420+0000","last_active":"2024-01-16T14:54:47.262723+0000","last_peered":"2024-01-16T14:54:47.262723+0000","last_clean":"2024-01-16T14:54:47.262723+0000","last_became_active":"2024-01-16T14:54:45.866022+0000","last_became_peered":"2024-01-16T14:54:45.866022+0000","last_unstale":"2024-01-16T14:54:47.262723+0000","last_undegraded":"2024-01-16T14:54:47.262723+0000","last_fullsized":"2024-01-16T14:54:47.262723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262181+0000","last_change":"2024-01-16T14:54:45.875523+0000","last_active":"2024-01-16T14:54:47.262181+0000","last_peered":"2024-01-16T14:54:47.262181+0000","last_clean":"2024-01-16T14:54:47.262181+0000","last_became_active":"2024-01-16T14:54:45.875252+0000","last_became_peered":"2024-01-16T14:54:45.875252+0000","last_unstale":"2024-01-16T14:54:47.262181+0000","last_undegraded":"2024-01-16T14:54:47.262181+0000","last_fullsized":"2024-01-16T14:54:47.262181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854401+0000","last_change":"2024-01-16T14:54:45.865807+0000","last_active":"2024-01-16T14:54:46.854401+0000","last_peered":"2024-01-16T14:54:46.854401+0000","last_clean":"2024-01-16T14:54:46.854401+0000","last_became_active":"2024-01-16T14:54:45.865257+0000","last_became_peered":"2024-01-16T14:54:45.865257+0000","last_unstale":"2024-01-16T14:54:46.854401+0000","last_undegraded":"2024-01-16T14:54:46.854401+0000","last_fullsized":"2024-01-16T14:54:46.854401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261310+0000","last_change":"2024-01-16T14:54:45.883032+0000","last_active":"2024-01-16T14:54:47.261310+0000","last_peered":"2024-01-16T14:54:47.261310+0000","last_clean":"2024-01-16T14:54:47.261310+0000","last_became_active":"2024-01-16T14:54:45.882768+0000","last_became_peered":"2024-01-16T14:54:45.882768+0000","last_unstale":"2024-01-16T14:54:47.261310+0000","last_undegraded":"2024-01-16T14:54:47.261310+0000","last_fullsized":"2024-01-16T14:54:47.261310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853768+0000","last_change":"2024-01-16T14:54:45.866167+0000","last_active":"2024-01-16T14:54:46.853768+0000","last_peered":"2024-01-16T14:54:46.853768+0000","last_clean":"2024-01-16T14:54:46.853768+0000","last_became_active":"2024-01-16T14:54:45.865393+0000","last_became_peered":"2024-01-16T14:54:45.865393+0000","last_unstale":"2024-01-16T14:54:46.853768+0000","last_undegraded":"2024-01-16T14:54:46.853768+0000","last_fullsized":"2024-01-16T14:54:46.853768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261572+0000","last_change":"2024-01-16T14:54:45.882612+0000","last_active":"2024-01-16T14:54:47.261572+0000","last_peered":"2024-01-16T14:54:47.261572+0000","last_clean":"2024-01-16T14:54:47.261572+0000","last_became_active":"2024-01-16T14:54:45.882301+0000","last_became_peered":"2024-01-16T14:54:45.882301+0000","last_unstale":"2024-01-16T14:54:47.261572+0000","last_undegraded":"2024-01-16T14:54:47.261572+0000","last_fullsized":"2024-01-16T14:54:47.261572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262090+0000","last_change":"2024-01-16T14:54:45.876292+0000","last_active":"2024-01-16T14:54:47.262090+0000","last_peered":"2024-01-16T14:54:47.262090+0000","last_clean":"2024-01-16T14:54:47.262090+0000","last_became_active":"2024-01-16T14:54:45.875995+0000","last_became_peered":"2024-01-16T14:54:45.875995+0000","last_unstale":"2024-01-16T14:54:47.262090+0000","last_undegraded":"2024-01-16T14:54:47.262090+0000","last_fullsized":"2024-01-16T14:54:47.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261512+0000","last_change":"2024-01-16T14:54:45.866707+0000","last_active":"2024-01-16T14:54:47.261512+0000","last_peered":"2024-01-16T14:54:47.261512+0000","last_clean":"2024-01-16T14:54:47.261512+0000","last_became_active":"2024-01-16T14:54:45.866333+0000","last_became_peered":"2024-01-16T14:54:45.866333+0000","last_unstale":"2024-01-16T14:54:47.261512+0000","last_undegraded":"2024-01-16T14:54:47.261512+0000","last_fullsized":"2024-01-16T14:54:47.261512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:54:50.493 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-16T14:54:50.493 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-16T14:54:50.494 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-16T14:54:50.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-16T14:54:50.494 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-16T14:54:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.686+0000 7f63dc4ae700 1 Processor -- start 2024-01-16T14:54:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.686+0000 7f63dc4ae700 1 -- start start 2024-01-16T14:54:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63dc4ae700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63d40b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63dc4ae700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d406dbe0 con 0x7f63d406b6e0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63dc4ae700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d41622e0 con 0x7f63d4161a90 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63dc4ae700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4147ca0 con 0x7f63d41478c0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63daa4b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f63d406b6e0 0x7f63d406e370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58332/0 (socket says 172.21.15.26:58332) 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.687+0000 7f63daa4b700 1 -- 172.21.15.26:0/2349879216 learned_addr learned my addr 172.21.15.26:0/2349879216 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63d40b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252398185 0 0) 0x7f63d406dbe0 con 0x7f63d406b6e0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c00035b0 con 0x7f63d406b6e0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688325129 0 0) 0x7f63d41622e0 con 0x7f63d4161a90 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d406dbe0 con 0x7f63d4161a90 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798154174 0 0) 0x7f63c00035b0 con 0x7f63d406b6e0 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 msgr2=0x7f63d40b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63d40b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 >> v1:172.21.15.93:6789/0 conn(0x7f63d4161a90 legacy=0x7f63d406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d41625c0 con 0x7f63d406b6e0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f63cc0019f0 con 0x7f63d406b6e0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63d40b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 >> v1:172.21.15.26:6789/0 conn(0x7f63d406b6e0 legacy=0x7f63d406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63c0003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c0004d90 0x7f63c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4147ca0 con 0x7f63c0003bc0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c000bac0 con 0x7f63c0004d90 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c000bc00 con 0x7f63d41478c0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63c0003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63d9a49700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.688+0000 7f63daa4b700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c0004d90 0x7f63c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 -- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 msgr2=0x7f63c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41478c0 0x7f63c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 -- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c0004d90 msgr2=0x7f63c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c0004d90 0x7f63c0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 -- 172.21.15.26:0/2349879216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80094a0 con 0x7f63c0003bc0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63da24a700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63c0003fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009d60 tx=0x7f63c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c480359a6887517 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800e9a0 con 0x7f63c0003bc0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c800eb00 con 0x7f63c0003bc0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63d9248700 1 -- 172.21.15.26:0/2349879216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800ee90 con 0x7f63c0003bc0 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63dc4ae700 1 -- 172.21.15.26:0/2349879216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 msgr2=0x7f63c0003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.689+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/2349879216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63c0003fc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009d60 tx=0x7f63c0005880 comp rx=0 tx=0).stop 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.690+0000 7f63daa4b700 1 -- 172.21.15.26:0/2349879216 reap_dead start 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.690+0000 7f63dc4ae700 1 -- 172.21.15.26:0/2349879216 shutdown_connections 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.690+0000 7f63dc4ae700 1 -- 172.21.15.26:0/2349879216 >> 172.21.15.26:0/2349879216 conn(0x7f63d40d0aa0 msgr2=0x7f63d40d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.690+0000 7f63dc4ae700 1 -- 172.21.15.26:0/2349879216 shutdown_connections 2024-01-16T14:54:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.690+0000 7f63dc4ae700 1 -- 172.21.15.26:0/2349879216 wait complete. 2024-01-16T14:54:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.691+0000 7f63dc4ae700 1 Processor -- start 2024-01-16T14:54:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.692+0000 7f63dc4ae700 1 -- start start 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.692+0000 7f63dc4ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.692+0000 7f63dc4ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 0x7f63d41f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 0x7f63d4143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4144290 con 0x7f63c0003bc0 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415f720 con 0x7f63d415ef40 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415fa00 con 0x7f63d41438c0 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49502/0 (socket says 172.21.15.26:49502) 2024-01-16T14:54:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 -- 172.21.15.26:0/124185142 learned_addr learned my addr 172.21.15.26:0/124185142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63d9a49700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 0x7f63d41f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63daa4b700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 0x7f63d4143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 -- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 msgr2=0x7f63d4143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 0x7f63d4143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 -- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 msgr2=0x7f63d41f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 0x7f63d41f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 -- 172.21.15.26:0/124185142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d415fce0 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63daa4b700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 0x7f63d4143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63d9a49700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 0x7f63d41f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63da24a700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63c80090b0 tx=0x7f63c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800e9a0 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8003ab0 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d415ffc0 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8003e60 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.693+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d415f3a0 con 0x7f63c0003bc0 2024-01-16T14:54:50.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.694+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d41604e0 con 0x7f63c0003bc0 2024-01-16T14:54:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.695+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f63c8013470 con 0x7f63c0003bc0 2024-01-16T14:54:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.695+0000 7f63c6ffd700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 0x7f63b806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.695+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c808b680 con 0x7f63c0003bc0 2024-01-16T14:54:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.695+0000 7f63d9a49700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 0x7f63b806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.696+0000 7f63d9a49700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 0x7f63b806d810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63d0000c00 tx=0x7f63d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.700+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63d41604e0 con 0x7f63c0003bc0 2024-01-16T14:54:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.976+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f63d40b5390 con 0x7f63c0003bc0 2024-01-16T14:54:50.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.979+0000 7f63c6ffd700 1 -- 172.21.15.26:0/124185142 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171554 (secure 0 0 0) 0x7f63c8056580 con 0x7f63c0003bc0 2024-01-16T14:54:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 msgr2=0x7f63b806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 0x7f63b806d810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63d0000c00 tx=0x7f63d0009040 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 msgr2=0x7f63d40b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63c80090b0 tx=0x7f63c80047c0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 shutdown_connections 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63d41438c0 0x7f63d4143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f63b806b350 0x7f63b806d810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63c0003bc0 0x7f63d40b17e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 --2- 172.21.15.26:0/124185142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63d415ef40 0x7f63d41f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 >> 172.21.15.26:0/124185142 conn(0x7f63d40d0aa0 msgr2=0x7f63d4142240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 shutdown_connections 2024-01-16T14:54:50.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:50.982+0000 7f63dc4ae700 1 -- 172.21.15.26:0/124185142 wait complete. 2024-01-16T14:54:50.979 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":190365378},{"type":"v1","addr":"172.21.15.93:6833","nonce":190365378}]},"active_addr":"172.21.15.93:6833/190365378","active_change":"2024-01-16T14:54:33.339502+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":720269274}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":1791003473}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":4228979579}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":2389296969}]}]} 2024-01-16T14:54:50.982 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-16T14:54:50.982 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-16T14:54:50.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.136+0000 7f45876d0700 1 Processor -- start 2024-01-16T14:54:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.136+0000 7f45876d0700 1 -- start start 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f45876d0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f45876d0700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 0x7f458006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f45876d0700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800af940 con 0x7f458015be40 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f45876d0700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45800b1130 con 0x7f458006b7c0 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f45876d0700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458006c1b0 con 0x7f45800d0ec0 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4585c6d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 0x7f458006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38334/0 (socket says 172.21.15.26:38334) 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 -- 172.21.15.26:0/2538920660 learned_addr learned my addr 172.21.15.26:0/2538920660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f458546c700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f458015be40 0x7f45800b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49622/0 (socket says 172.21.15.26:49622) 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4577fff700 1 -- 172.21.15.26:0/2538920660 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381531855 0 0) 0x7f457c001110 con 0x7f458015be40 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4577fff700 1 -- 172.21.15.26:0/2538920660 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f456c0035b0 con 0x7f458015be40 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 -- 172.21.15.26:0/2538920660 >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 msgr2=0x7f458006bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 --2- 172.21.15.26:0/2538920660 >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 0x7f458006bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 -- 172.21.15.26:0/2538920660 >> v1:172.21.15.93:6789/0 conn(0x7f458015be40 legacy=0x7f45800b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4584c6b700 1 -- 172.21.15.26:0/2538920660 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800b0ab0 con 0x7f45800d0ec0 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.137+0000 7f4585c6d700 1 --2- 172.21.15.26:0/2538920660 >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 0x7f458006bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f4584c6b700 1 --2- 172.21.15.26:0/2538920660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45700098f0 tx=0x7f4570009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1350c461ea0c551 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f4577fff700 1 -- 172.21.15.26:0/2538920660 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570011830 con 0x7f45800d0ec0 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f4577fff700 1 -- 172.21.15.26:0/2538920660 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4570011990 con 0x7f45800d0ec0 2024-01-16T14:54:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f4577fff700 1 -- 172.21.15.26:0/2538920660 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570011d20 con 0x7f45800d0ec0 2024-01-16T14:54:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 -- 172.21.15.26:0/2538920660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 msgr2=0x7f45800af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 --2- 172.21.15.26:0/2538920660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45700098f0 tx=0x7f4570009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 -- 172.21.15.26:0/2538920660 shutdown_connections 2024-01-16T14:54:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 --2- 172.21.15.26:0/2538920660 >> v2:172.21.15.93:3300/0 conn(0x7f458006b7c0 0x7f458006bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 --2- 172.21.15.26:0/2538920660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f45800af400 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.138+0000 7f45876d0700 1 -- 172.21.15.26:0/2538920660 >> 172.21.15.26:0/2538920660 conn(0x7f45800717f0 msgr2=0x7f45800b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.139+0000 7f45876d0700 1 -- 172.21.15.26:0/2538920660 shutdown_connections 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.139+0000 7f45876d0700 1 -- 172.21.15.26:0/2538920660 wait complete. 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.140+0000 7f45876d0700 1 Processor -- start 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.140+0000 7f45876d0700 1 -- start start 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f458006b7c0 0x7f4580159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f4580157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45801534b0 con 0x7f458015be40 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800aeaf0 con 0x7f458006b7c0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f45876d0700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45800adc00 con 0x7f45800d0ec0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49504/0 (socket says 172.21.15.26:49504) 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 -- 172.21.15.26:0/1241546301 learned_addr learned my addr 172.21.15.26:0/1241546301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f458546c700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f458006b7c0 0x7f4580159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4584c6b700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f4580157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 -- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 msgr2=0x7f4580157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f4580157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 -- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f458006b7c0 msgr2=0x7f4580159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f458006b7c0 0x7f4580159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 -- 172.21.15.26:0/1241546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4578009a70 con 0x7f458015be40 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.141+0000 7f4585c6d700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f457800efc0 tx=0x7f457800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.142+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457800ca90 con 0x7f458015be40 2024-01-16T14:54:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.142+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457800f040 con 0x7f458015be40 2024-01-16T14:54:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.142+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457800ce10 con 0x7f458015be40 2024-01-16T14:54:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.142+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4570009550 con 0x7f458015be40 2024-01-16T14:54:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.142+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45800ae4a0 con 0x7f458015be40 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.143+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45800aeca0 con 0x7f458015be40 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.143+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4578014480 con 0x7f458015be40 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.144+0000 7f4575ffb700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 0x7f456406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.144+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f457808a190 con 0x7f458015be40 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.144+0000 7f458546c700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 0x7f456406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.144+0000 7f458546c700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 0x7f456406d840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f457c001370 tx=0x7f457c007c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.149+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45800aeca0 con 0x7f458015be40 2024-01-16T14:54:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.362+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45800aeca0 con 0x7f458015be40 2024-01-16T14:54:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.363+0000 7f4575ffb700 1 -- 172.21.15.26:0/1241546301 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f45800aeca0 con 0x7f458015be40 2024-01-16T14:54:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:51.346 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 msgr2=0x7f456406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 0x7f456406d840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f457c001370 tx=0x7f457c007c40 comp rx=0 tx=0).stop 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 msgr2=0x7f45801f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f457800efc0 tx=0x7f457800c3a0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 shutdown_connections 2024-01-16T14:54:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f456406b380 0x7f456406d840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f458015be40 0x7f45801f6d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f458006b7c0 0x7f4580159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 --2- 172.21.15.26:0/1241546301 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45800d0ec0 0x7f4580157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 >> 172.21.15.26:0/1241546301 conn(0x7f45800717f0 msgr2=0x7f458006ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 shutdown_connections 2024-01-16T14:54:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.366+0000 7f45876d0700 1 -- 172.21.15.26:0/1241546301 wait complete. 2024-01-16T14:54:51.363 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-16T14:54:51.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:51.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.517+0000 7fb3664b9700 1 Processor -- start 2024-01-16T14:54:51.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.517+0000 7fb3664b9700 1 -- start start 2024-01-16T14:54:51.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb3664b9700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 0x7fb360072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb3664b9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb360072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb3664b9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36015f3e0 con 0x7fb36006bac0 2024-01-16T14:54:51.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb3664b9700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3600affd0 con 0x7fb3600726a0 2024-01-16T14:54:51.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb3664b9700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb360073500 con 0x7fb3600b3d90 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb364a56700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb36006bac0 0x7fb36006bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58340/0 (socket says 172.21.15.26:58340) 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb364a56700 1 -- 172.21.15.26:0/3756864273 learned_addr learned my addr 172.21.15.26:0/3756864273 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb35ffff700 1 --2- 172.21.15.26:0/3756864273 >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 0x7fb360072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.518+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/3756864273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb360072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230025073 0 0) 0x7fb36015f3e0 con 0x7fb36006bac0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3480035b0 con 0x7fb36006bac0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35f7fe700 1 -- 172.21.15.26:0/3756864273 >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 msgr2=0x7fb360072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173161807 0 0) 0x7fb3480035b0 con 0x7fb36006bac0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/3756864273 >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 0x7fb360072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35f7fe700 1 -- 172.21.15.26:0/3756864273 >> v1:172.21.15.26:6789/0 conn(0x7fb36006bac0 legacy=0x7fb36006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35f7fe700 1 -- 172.21.15.26:0/3756864273 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36015e7d0 con 0x7fb3600b3d90 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35ffff700 1 --2- 172.21.15.26:0/3756864273 >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 0x7fb360072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/3756864273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb360072fc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb354009840 tx=0x7fb354009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3e55c17de6faca8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3580019f0 con 0x7fb3600b3d90 2024-01-16T14:54:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354004030 con 0x7fb3600b3d90 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.519+0000 7fb35effd700 1 -- 172.21.15.26:0/3756864273 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3540043c0 con 0x7fb3600b3d90 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 -- 172.21.15.26:0/3756864273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 msgr2=0x7fb360072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 --2- 172.21.15.26:0/3756864273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb360072fc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb354009840 tx=0x7fb354009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 -- 172.21.15.26:0/3756864273 shutdown_connections 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 --2- 172.21.15.26:0/3756864273 >> v2:172.21.15.93:3300/0 conn(0x7fb3600726a0 0x7fb360072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 --2- 172.21.15.26:0/3756864273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb360072fc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 -- 172.21.15.26:0/3756864273 >> 172.21.15.26:0/3756864273 conn(0x7fb3600717f0 msgr2=0x7fb3600b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 -- 172.21.15.26:0/3756864273 shutdown_connections 2024-01-16T14:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.520+0000 7fb3664b9700 1 -- 172.21.15.26:0/3756864273 wait complete. 2024-01-16T14:54:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.521+0000 7fb3664b9700 1 Processor -- start 2024-01-16T14:54:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 -- start start 2024-01-16T14:54:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3600726a0 0x7fb3601f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb36014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600af010 con 0x7fb36006bac0 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600af2f0 con 0x7fb3600726a0 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb3664b9700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb360159830 con 0x7fb3600b3d90 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb35ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb35ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49506/0 (socket says 172.21.15.26:49506) 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.522+0000 7fb35ffff700 1 -- 172.21.15.26:0/807261452 learned_addr learned my addr 172.21.15.26:0/807261452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb364a56700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb36014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3600726a0 0x7fb3601f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 -- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 msgr2=0x7fb36014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb36014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 -- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3600726a0 msgr2=0x7fb3601f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3600726a0 0x7fb3601f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 -- 172.21.15.26:0/807261452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500094a0 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35ffff700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ebe0 tx=0x7fb350009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000c910 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35000ca70 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3540094a0 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35000cde0 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.523+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3600aec50 con 0x7fb36006bac0 2024-01-16T14:54:51.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.524+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb360159d50 con 0x7fb36006bac0 2024-01-16T14:54:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.525+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb350013470 con 0x7fb36006bac0 2024-01-16T14:54:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.525+0000 7fb35cff9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 0x7fb34c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.525+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb35008a110 con 0x7fb36006bac0 2024-01-16T14:54:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.525+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 0x7fb34c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.526+0000 7fb35f7fe700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 0x7fb34c06d810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3540090b0 tx=0x7fb354008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.530+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb360159d50 con 0x7fb36006bac0 2024-01-16T14:54:51.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.744+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb360159d50 con 0x7fb36006bac0 2024-01-16T14:54:51.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.745+0000 7fb35cff9700 1 -- 172.21.15.26:0/807261452 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb360159d50 con 0x7fb36006bac0 2024-01-16T14:54:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:51.729 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.747+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 msgr2=0x7fb34c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.747+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 0x7fb34c06d810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3540090b0 tx=0x7fb354008030 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.747+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 msgr2=0x7fb3601501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.747+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb35000ebe0 tx=0x7fb350009ec0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 shutdown_connections 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3600b3d90 0x7fb36014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb34c06b350 0x7fb34c06d810 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb36006bac0 0x7fb3601501f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 --2- 172.21.15.26:0/807261452 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3600726a0 0x7fb3601f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 >> 172.21.15.26:0/807261452 conn(0x7fb3600717f0 msgr2=0x7fb36006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:51.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 shutdown_connections 2024-01-16T14:54:51.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:51.748+0000 7fb3664b9700 1 -- 172.21.15.26:0/807261452 wait complete. 2024-01-16T14:54:51.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-16T14:54:51.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-16T14:54:51.746 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-16T14:54:51.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-16T14:54:51.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-16T14:54:51.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-16T14:54:51.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-16T14:54:51.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-16T14:54:51.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.003+0000 7f410c973700 1 Processor -- start 2024-01-16T14:54:51.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.003+0000 7f410c973700 1 -- start start 2024-01-16T14:54:51.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.004+0000 7f410c973700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4108070af0 0x7f410806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.004+0000 7f410c973700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080ca690 con 0x7f410816ed10 2024-01-16T14:54:51.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.004+0000 7f410c973700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080ca970 con 0x7f4108070fb0 2024-01-16T14:54:51.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.004+0000 7f410c973700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080cb030 con 0x7f4108070af0 2024-01-16T14:54:51.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.005+0000 7fb66abfb700 1 Processor -- start 2024-01-16T14:54:51.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.006+0000 7f4105d9b700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f4108070fb0 0x7f410816e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49634/0 (socket says 172.21.15.26:49634) 2024-01-16T14:54:51.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.006+0000 7f4105d9b700 1 -- 172.21.15.26:0/3907890172 learned_addr learned my addr 172.21.15.26:0/3907890172 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:51.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f9fedc5a700 1 Processor -- start 2024-01-16T14:54:51.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.006+0000 7f410659c700 1 --2- 172.21.15.26:0/3907890172 >> v2:172.21.15.26:3300/0 conn(0x7f4108070af0 0x7f410806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030312658 0 0) 0x7f41080ca690 con 0x7f410816ed10 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f00035b0 con 0x7f410816ed10 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1653155254 0 0) 0x7f40f00035b0 con 0x7f410816ed10 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 >> v2:172.21.15.26:3300/0 conn(0x7f4108070af0 msgr2=0x7f410806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 --2- 172.21.15.26:0/3907890172 >> v2:172.21.15.26:3300/0 conn(0x7f4108070af0 0x7f410806cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 >> v1:172.21.15.93:6789/0 conn(0x7f4108070fb0 legacy=0x7f410816e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080cac50 con 0x7f410816ed10 2024-01-16T14:54:51.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f40f8001670 con 0x7f410816ed10 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 >> v1:172.21.15.26:6789/0 conn(0x7f410816ed10 legacy=0x7f41080cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40f0003a00 0x7f40f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f40f0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f0005210 0x7f40f00079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41080cb030 con 0x7f40f0005210 2024-01-16T14:54:51.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00081d0 con 0x7f40f0004850 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0008390 con 0x7f40f0003a00 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4106d9d700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f0005210 0x7f40f00079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f410659c700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40f0003a00 0x7f40f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f40f0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 -- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40f0003a00 msgr2=0x7f40f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40f0003a00 0x7f40f0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 -- 172.21.15.26:0/3907890172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f0005210 msgr2=0x7f40f00079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f0005210 0x7f40f00079f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4105d9b700 1 -- 172.21.15.26:0/3907890172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080cac50 con 0x7f40f0004850 2024-01-16T14:54:51.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f4106d9d700 1 -- 172.21.15.26:0/3907890172 reap_dead start 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.005+0000 7fb66abfb700 1 -- start start 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb664071390 0x7fb66415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb66415a520 0x7fb66415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6640ce000 con 0x7fb66415a520 2024-01-16T14:54:51.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6640ce2c0 con 0x7fb664071390 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb66abfb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640ce5a0 con 0x7fb66415db10 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38370/0 (socket says 172.21.15.26:38370) 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 -- 172.21.15.26:0/2474922631 learned_addr learned my addr 172.21.15.26:0/2474922631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb669198700 1 --2- 172.21.15.26:0/2474922631 >> v2:172.21.15.26:3300/0 conn(0x7fb66415a520 0x7fb66415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 -- 172.21.15.26:0/2474922631 >> v2:172.21.15.93:3300/0 conn(0x7fb664071390 msgr2=0x7fb66415e430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 --2- 172.21.15.26:0/2474922631 >> v2:172.21.15.93:3300/0 conn(0x7fb664071390 0x7fb66415e430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 -- 172.21.15.26:0/2474922631 >> v2:172.21.15.26:3300/0 conn(0x7fb66415a520 msgr2=0x7fb66415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 --2- 172.21.15.26:0/2474922631 >> v2:172.21.15.26:3300/0 conn(0x7fb66415a520 0x7fb66415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.007+0000 7fb668997700 1 -- 172.21.15.26:0/2474922631 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6640ce880 con 0x7fb66415db10 2024-01-16T14:54:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7fb668997700 1 --2- 172.21.15.26:0/2474922631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb664069600 tx=0x7fb65800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad9ac46a6158550 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:51.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.008+0000 7f9fedc5a700 1 -- start start 2024-01-16T14:54:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f1e77a22700 1 Processor -- start 2024-01-16T14:54:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7fb6637fe700 1 -- 172.21.15.26:0/2474922631 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658010070 con 0x7fb66415db10 2024-01-16T14:54:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7fb6637fe700 1 -- 172.21.15.26:0/2474922631 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65800d9d0 con 0x7fb66415db10 2024-01-16T14:54:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7fb6637fe700 1 -- 172.21.15.26:0/2474922631 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65800dd60 con 0x7fb66415db10 2024-01-16T14:54:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f4105d9b700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f40f0004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005250 tx=0x7f40f0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd725fe7ca27472c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f9fedc5a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9fe80d04a0 0x7f9fe815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f9fedc5a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80ce000 con 0x7f9fe815b170 2024-01-16T14:54:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f9fedc5a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80cb2d0 con 0x7f9fe8071390 2024-01-16T14:54:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f9fedc5a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe80ce2c0 con 0x7f9fe80d04a0 2024-01-16T14:54:52.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.009+0000 7f1e77a22700 1 -- start start 2024-01-16T14:54:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9fe80d04a0 0x7f9fe815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9fe80d04a0 0x7f9fe815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47550/0 (socket says 172.21.15.26:47550) 2024-01-16T14:54:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4216899948 learned_addr learned my addr 172.21.15.26:0/4216899948 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4216899948 >> v1:172.21.15.93:6789/0 conn(0x7f9fe8071390 legacy=0x7f9fe815a520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4216899948 >> v1:172.21.15.26:6789/0 conn(0x7f9fe815b170 legacy=0x7f9fe815db10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4216899948 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80ce5a0 con 0x7f9fe80d04a0 2024-01-16T14:54:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc005b60 con 0x7f40f0004850 2024-01-16T14:54:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc005cc0 con 0x7f40f0004850 2024-01-16T14:54:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7f410559a700 1 -- 172.21.15.26:0/3907890172 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc007310 con 0x7f40f0004850 2024-01-16T14:54:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 -- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 msgr2=0x7f40f0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f40f0004cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40fc005250 tx=0x7f40f0005880 comp rx=0 tx=0).stop 2024-01-16T14:54:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.010+0000 7fc50b14a700 1 Processor -- start 2024-01-16T14:54:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 -- 172.21.15.26:0/2474922631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 msgr2=0x7fb66415def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 --2- 172.21.15.26:0/2474922631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb664069600 tx=0x7fb65800d330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 -- 172.21.15.26:0/3907890172 shutdown_connections 2024-01-16T14:54:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 --2- 172.21.15.26:0/3907890172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f40f0004cd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 -- 172.21.15.26:0/3907890172 >> 172.21.15.26:0/3907890172 conn(0x7f41080cf980 msgr2=0x7f41080cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 -- 172.21.15.26:0/2474922631 shutdown_connections 2024-01-16T14:54:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 --2- 172.21.15.26:0/2474922631 >> v2:172.21.15.26:3300/0 conn(0x7fb66415a520 0x7fb66415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 --2- 172.21.15.26:0/2474922631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415db10 0x7fb66415def0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 --2- 172.21.15.26:0/2474922631 >> v2:172.21.15.93:3300/0 conn(0x7fb664071390 0x7fb66415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fb66abfb700 1 -- 172.21.15.26:0/2474922631 >> 172.21.15.26:0/2474922631 conn(0x7fb6640cf980 msgr2=0x7fb6640cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 -- 172.21.15.26:0/3907890172 shutdown_connections 2024-01-16T14:54:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e77a22700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1e7006d400 0x7f1e700ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e77a22700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700ae520 con 0x7f1e700adda0 2024-01-16T14:54:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e77a22700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700ae800 con 0x7f1e70147700 2024-01-16T14:54:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e77a22700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700aeae0 con 0x7f1e7006d400 2024-01-16T14:54:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7fc50b14a700 1 -- start start 2024-01-16T14:54:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e74fbd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1e7006d400 0x7f1e700ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e74fbd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1e7006d400 0x7f1e700ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47560/0 (socket says 172.21.15.26:47560) 2024-01-16T14:54:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f1e74fbd700 1 -- 172.21.15.26:0/3480032418 learned_addr learned my addr 172.21.15.26:0/3480032418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f9fdeffd700 1 --2- 172.21.15.26:0/4216899948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe80d04a0 0x7f9fe815ac30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009920 tx=0x7f9fd0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae2aa253296624d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f9fe6ffd700 1 -- 172.21.15.26:0/4216899948 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0004230 con 0x7f9fe80d04a0 2024-01-16T14:54:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f9fe6ffd700 1 -- 172.21.15.26:0/4216899948 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0004390 con 0x7f9fe80d04a0 2024-01-16T14:54:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f9fe6ffd700 1 -- 172.21.15.26:0/4216899948 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0009e70 con 0x7f9fe80d04a0 2024-01-16T14:54:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc50b14a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc50414bf10 0x7fc504070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc50b14a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50406d1f0 con 0x7fc504165f70 2024-01-16T14:54:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc50b14a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5040cc080 con 0x7fc5040cbb80 2024-01-16T14:54:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc50b14a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5040cdf60 con 0x7fc50414bf10 2024-01-16T14:54:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e74fbd700 1 -- 172.21.15.26:0/3480032418 >> v1:172.21.15.93:6789/0 conn(0x7f1e70147700 legacy=0x7f1e7006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e74fbd700 1 -- 172.21.15.26:0/3480032418 >> v1:172.21.15.26:6789/0 conn(0x7f1e700adda0 legacy=0x7f1e70070d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e74fbd700 1 -- 172.21.15.26:0/3480032418 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700aedc0 con 0x7f1e7006d400 2024-01-16T14:54:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc508ee6700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc50414bf10 0x7fc504070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc508ee6700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc50414bf10 0x7fc504070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47576/0 (socket says 172.21.15.26:47576) 2024-01-16T14:54:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7fc508ee6700 1 -- 172.21.15.26:0/3752346831 learned_addr learned my addr 172.21.15.26:0/3752346831 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e74fbd700 1 --2- 172.21.15.26:0/3480032418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e7006d400 0x7f1e700ad860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098f0 tx=0x7f1e60009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a7dce84c90d8756 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.013+0000 7fc508ee6700 1 -- 172.21.15.26:0/3752346831 >> v1:172.21.15.93:6789/0 conn(0x7fc5040cbb80 legacy=0x7fc50406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.013+0000 7fc508ee6700 1 -- 172.21.15.26:0/3752346831 >> v1:172.21.15.26:6789/0 conn(0x7fc504165f70 legacy=0x7fc5040ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.013+0000 7fc508ee6700 1 -- 172.21.15.26:0/3752346831 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50406e950 con 0x7fc50414bf10 2024-01-16T14:54:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e67fff700 1 -- 172.21.15.26:0/3480032418 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004230 con 0x7f1e7006d400 2024-01-16T14:54:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e67fff700 1 -- 172.21.15.26:0/3480032418 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60004390 con 0x7f1e7006d400 2024-01-16T14:54:52.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.012+0000 7f1e67fff700 1 -- 172.21.15.26:0/3480032418 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600036a0 con 0x7f1e7006d400 2024-01-16T14:54:52.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.013+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4216899948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe80d04a0 msgr2=0x7f9fe815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.013+0000 7f9fedc5a700 1 --2- 172.21.15.26:0/4216899948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe80d04a0 0x7f9fe815ac30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009920 tx=0x7f9fd0009c30 comp rx=0 tx=0).stop 2024-01-16T14:54:52.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7fc508ee6700 1 --2- 172.21.15.26:0/3752346831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc50414bf10 0x7fc504070a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009e50 tx=0x7fc4fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf90d485fca0d26c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.011+0000 7f410c973700 1 -- 172.21.15.26:0/3907890172 wait complete. 2024-01-16T14:54:52.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f410c973700 1 Processor -- start 2024-01-16T14:54:52.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4216899948 shutdown_connections 2024-01-16T14:54:52.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f9fedc5a700 1 --2- 172.21.15.26:0/4216899948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe80d04a0 0x7f9fe815ac30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4216899948 >> 172.21.15.26:0/4216899948 conn(0x7f9fe80cf980 msgr2=0x7f9fe80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 -- 172.21.15.26:0/3480032418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e7006d400 msgr2=0x7f1e700ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 --2- 172.21.15.26:0/3480032418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e7006d400 0x7f1e700ad860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098f0 tx=0x7f1e60009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4216899948 shutdown_connections 2024-01-16T14:54:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7fc5037fe700 1 -- 172.21.15.26:0/3752346831 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00e070 con 0x7fc50414bf10 2024-01-16T14:54:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7fc5037fe700 1 -- 172.21.15.26:0/3752346831 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc003a20 con 0x7fc50414bf10 2024-01-16T14:54:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7fc5037fe700 1 -- 172.21.15.26:0/3752346831 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc003dd0 con 0x7fc50414bf10 2024-01-16T14:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 -- 172.21.15.26:0/3480032418 shutdown_connections 2024-01-16T14:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 --2- 172.21.15.26:0/3480032418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e7006d400 0x7f1e700ad860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 -- 172.21.15.26:0/3480032418 >> 172.21.15.26:0/3480032418 conn(0x7f1e700cfa10 msgr2=0x7f1e700cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 -- 172.21.15.26:0/3480032418 shutdown_connections 2024-01-16T14:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7fb66abfb700 1 -- 172.21.15.26:0/2474922631 shutdown_connections 2024-01-16T14:54:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4216899948 wait complete. 2024-01-16T14:54:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fc50b14a700 1 -- 172.21.15.26:0/3752346831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc50414bf10 msgr2=0x7fc504070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fc50b14a700 1 --2- 172.21.15.26:0/3752346831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc50414bf10 0x7fc504070a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009e50 tx=0x7fc4fc00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fc50b14a700 1 -- 172.21.15.26:0/3752346831 shutdown_connections 2024-01-16T14:54:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fc50b14a700 1 --2- 172.21.15.26:0/3752346831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc50414bf10 0x7fc504070a80 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009e50 tx=0x7fc4fc00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fc50b14a700 1 -- 172.21.15.26:0/3752346831 >> 172.21.15.26:0/3752346831 conn(0x7fc5040cf980 msgr2=0x7fc5040cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f410c973700 1 -- start start 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f4108164a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4108164fd0 0x7f4108165810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 0x7f4108165d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081639f0 con 0x7f41080b3e10 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4108163cd0 con 0x7f40f0004850 2024-01-16T14:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f410c973700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4108163fb0 con 0x7f4108164fd0 2024-01-16T14:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7f4106d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 0x7f4108165d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 0x7f4108165d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49558/0 (socket says 172.21.15.26:49558) 2024-01-16T14:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 -- 172.21.15.26:0/2540009275 learned_addr learned my addr 172.21.15.26:0/2540009275 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.015+0000 7fb66abfb700 1 -- 172.21.15.26:0/2474922631 wait complete. 2024-01-16T14:54:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f410659c700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f4108164a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f9fedc5a700 1 Processor -- start 2024-01-16T14:54:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4105d9b700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4108164fd0 0x7f4108165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.014+0000 7f1e77a22700 1 -- 172.21.15.26:0/3480032418 wait complete. 2024-01-16T14:54:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f1e77a22700 1 Processor -- start 2024-01-16T14:54:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 -- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4108164fd0 msgr2=0x7f4108165810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4108164fd0 0x7f4108165810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 -- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 msgr2=0x7f4108164a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40f0004850 0x7f4108164a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f4106d9d700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108164290 con 0x7f41080b3e10 2024-01-16T14:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7fb66abfb700 1 Processor -- start 2024-01-16T14:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f4106d9d700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 0x7f4108165d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41080c9f80 tx=0x7f40f800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7fb66abfb700 1 -- start start 2024-01-16T14:54:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.016+0000 7f9fedc5a700 1 -- start start 2024-01-16T14:54:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7fc50b14a700 1 -- 172.21.15.26:0/3752346831 shutdown_connections 2024-01-16T14:54:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9fe8071390 0x7f9fe815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 0x7f9fe806e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe815b170 0x7f9fe813aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80b3070 con 0x7f9fe80d04a0 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe815c3f0 con 0x7f9fe815b170 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fedc5a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe814f380 con 0x7f9fe8071390 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f9fdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 0x7f9fe806e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 0x7f9fe806e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49574/0 (socket says 172.21.15.26:49574) 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4035208602 learned_addr learned my addr 172.21.15.26:0/4035208602 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9fe8071390 msgr2=0x7f9fe815beb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9fe8071390 0x7f9fe815beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe815b170 msgr2=0x7f9fe813aaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fe815b170 0x7f9fe813aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009550 con 0x7f9fe80d04a0 2024-01-16T14:54:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f1e77a22700 1 -- start start 2024-01-16T14:54:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7006d400 0x7f1e700cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e700adda0 0x7f1e701401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1e70147700 0x7f1e700b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70140710 con 0x7f1e7006d400 2024-01-16T14:54:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700b06b0 con 0x7f1e700adda0 2024-01-16T14:54:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f1e77a22700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700afaa0 con 0x7f1e70147700 2024-01-16T14:54:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7f9fdeffd700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 0x7f9fe806e580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006fd0 tx=0x7f9fd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80052e0 con 0x7f41080b3e10 2024-01-16T14:54:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8005460 con 0x7f41080b3e10 2024-01-16T14:54:52.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80038c0 con 0x7f41080b3e10 2024-01-16T14:54:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4108164570 con 0x7f41080b3e10 2024-01-16T14:54:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.017+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41080b5260 con 0x7f41080b3e10 2024-01-16T14:54:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41080b5540 con 0x7f41080b3e10 2024-01-16T14:54:52.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fc50b14a700 1 -- 172.21.15.26:0/3752346831 wait complete. 2024-01-16T14:54:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e74fbd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e700adda0 0x7f1e701401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e74fbd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e700adda0 0x7f1e701401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47606/0 (socket says 172.21.15.26:47606) 2024-01-16T14:54:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e74fbd700 1 -- 172.21.15.26:0/2497122540 learned_addr learned my addr 172.21.15.26:0/2497122540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7006d400 0x7f1e700cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e75fbf700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1e70147700 0x7f1e700b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 0x7fb6640b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415a520 0x7fb6640b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb66415db10 0x7fb6640b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640b5790 con 0x7fb66415db10 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640b54b0 con 0x7fb664071390 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb66abfb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66415b320 con 0x7fb66415a520 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb668997700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 0x7fb6640b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb668997700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 0x7fb6640b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47612/0 (socket says 172.21.15.26:47612) 2024-01-16T14:54:52.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb668997700 1 -- 172.21.15.26:0/3683572429 learned_addr learned my addr 172.21.15.26:0/3683572429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.018+0000 7fb663fff700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415a520 0x7fb6640b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 -- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415a520 msgr2=0x7fb6640b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb66415a520 0x7fb6640b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 -- 172.21.15.26:0/3683572429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb66415db10 msgr2=0x7fb6640b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb66415db10 0x7fb6640b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580085d0 con 0x7fb664071390 2024-01-16T14:54:52.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb668997700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 0x7fb6640b29b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb65800d370 tx=0x7fb658004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 -- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1e70147700 msgr2=0x7f1e700b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1e70147700 0x7f1e700b0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 -- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e700adda0 msgr2=0x7f1e701401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1e700adda0 0x7f1e701401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f1e757be700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009550 con 0x7f1e7006d400 2024-01-16T14:54:52.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00036a0 con 0x7f9fe80d04a0 2024-01-16T14:54:52.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0003800 con 0x7f9fe80d04a0 2024-01-16T14:54:52.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0003b90 con 0x7f9fe80d04a0 2024-01-16T14:54:52.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.019+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580044d0 con 0x7fb664071390 2024-01-16T14:54:52.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7f1e757be700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7006d400 0x7f1e700cd350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800be20 tx=0x7f1e68009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fc50b14a700 1 Processor -- start 2024-01-16T14:54:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fc50b14a700 1 -- start start 2024-01-16T14:54:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc5040cbb80 0x7fc504163fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc504165f70 0x7fc5041f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 0x7fc504164db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5041652f0 con 0x7fc504165f70 2024-01-16T14:54:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504164500 con 0x7fc5040cbb80 2024-01-16T14:54:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc50b14a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50413faa0 con 0x7fc504164920 2024-01-16T14:54:52.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66415b600 con 0x7fb664071390 2024-01-16T14:54:52.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66413c6d0 con 0x7fb664071390 2024-01-16T14:54:52.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6580036a0 con 0x7fb664071390 2024-01-16T14:54:52.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.020+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658003a50 con 0x7fb664071390 2024-01-16T14:54:52.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.021+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb644000fc0 con 0x7fb664071390 2024-01-16T14:54:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7f1bf397f700 1 Processor -- start 2024-01-16T14:54:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.021+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe80b2740 con 0x7f9fe80d04a0 2024-01-16T14:54:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.021+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe80b2d00 con 0x7f9fe80d04a0 2024-01-16T14:54:52.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fe80ceda0 con 0x7f9fe80d04a0 2024-01-16T14:54:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f40f8003a20 con 0x7f41080b3e10 2024-01-16T14:54:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7f40f6ffd700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40e806b420 0x7f40e806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40f8089a80 con 0x7f41080b3e10 2024-01-16T14:54:52.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7f40f6ffd700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] conn(0x7f40e8073640 0x7f40e8075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40e8076110 con 0x7f40e8073640 2024-01-16T14:54:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40f8089e30 con 0x7f41080b3e10 2024-01-16T14:54:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 0x7fc504164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 0x7fc504164db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38434/0 (socket says 172.21.15.26:38434) 2024-01-16T14:54:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 -- 172.21.15.26:0/435607634 learned_addr learned my addr 172.21.15.26:0/435607634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc508ee6700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc5040cbb80 0x7fc504163fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7f1bf397f700 1 -- start start 2024-01-16T14:54:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 -- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc5040cbb80 msgr2=0x7fc504163fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc5040cbb80 0x7fc504163fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 -- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc504165f70 msgr2=0x7fc5041f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc504165f70 0x7fc5041f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.022+0000 7fc5096e7700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc009b30 con 0x7fc504164920 2024-01-16T14:54:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7fc5096e7700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 0x7fc504164db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c3c0 tx=0x7fc4f800c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb658003bb0 con 0x7fb664071390 2024-01-16T14:54:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf397f700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bec15c010 0x7f1bec0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf397f700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1bec0cf120 0x7f1bec06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf397f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec06f610 con 0x7f1bec0ce560 2024-01-16T14:54:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf397f700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0cd7f0 con 0x7f1bec15c010 2024-01-16T14:54:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf397f700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0cdac0 con 0x7f1bec0cf120 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.023+0000 7fb6617fa700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb65006b350 0x7fb65006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb65808b530 con 0x7fb664071390 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb6617fa700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] conn(0x7fb6500734f0 0x7fb650075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb650075fc0 con 0x7fb6500734f0 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb65808db10 con 0x7fb664071390 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb663fff700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb65006b350 0x7fb65006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80052d0 con 0x7fc504164920 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8005450 con 0x7fc504164920 2024-01-16T14:54:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50413fd80 con 0x7fc504164920 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80038c0 con 0x7fc504164920 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc504140340 con 0x7fc504164920 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800cc10 con 0x7f1e7006d400 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800cd70 con 0x7f1e7006d400 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68014680 con 0x7f1e7006d400 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7fb669198700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] conn(0x7fb6500734f0 0x7fb650075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7fb669198700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] conn(0x7fb6500734f0 0x7fb650075910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf171b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bec15c010 0x7f1bec0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf171b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bec15c010 0x7f1bec0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49612/0 (socket says 172.21.15.26:49612) 2024-01-16T14:54:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.024+0000 7f1bf171b700 1 -- 172.21.15.26:0/4086321882 learned_addr learned my addr 172.21.15.26:0/4086321882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc50406e620 con 0x7fc504164920 2024-01-16T14:54:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7fb663fff700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb65006b350 0x7fb65006d810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb66415d8d0 tx=0x7fb65c00d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f1bf171b700 1 -- 172.21.15.26:0/4086321882 >> v2:172.21.15.93:3300/0 conn(0x7f1bec0cf120 msgr2=0x7f1bec06faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f1bf171b700 1 --2- 172.21.15.26:0/4086321882 >> v2:172.21.15.93:3300/0 conn(0x7f1bec0cf120 0x7f1bec06faa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f1bf171b700 1 -- 172.21.15.26:0/4086321882 >> v1:172.21.15.26:6789/0 conn(0x7f1bec0ce560 legacy=0x7f1bec0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f1bf171b700 1 -- 172.21.15.26:0/4086321882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec06ece0 con 0x7f1bec15c010 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== osd.5 v2:172.21.15.93:6823/1651349923 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb650075fc0 con 0x7fb6500734f0 2024-01-16T14:54:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9fd0022070 con 0x7f9fe80d04a0 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe4ff9700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9fc806b420 0x7f9fc806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd008bb00 con 0x7f9fe80d04a0 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe77fe700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9fc806b420 0x7f9fc806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe4ff9700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] conn(0x7f9fc8073640 0x7f9fc8075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fc8076110 con 0x7f9fc8073640 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd008beb0 con 0x7f9fe80d04a0 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f9fe7fff700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] conn(0x7f9fc8073640 0x7f9fc8075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.025+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700afd80 con 0x7f1e7006d400 2024-01-16T14:54:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700b0340 con 0x7f1e7006d400 2024-01-16T14:54:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7f9fe77fe700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9fc806b420 0x7f9fc806d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fe816d4b0 tx=0x7f9fd8006c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7f410659c700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40e806b420 0x7f40e806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7f1bf171b700 1 --2- 172.21.15.26:0/4086321882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec0cd2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0b7d50 tx=0x7f1be8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e082fa75ecc679 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7f4105d9b700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] conn(0x7f40e8073640 0x7f40e8075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.026+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc4f8003a20 con 0x7fc504164920 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc5017fa700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc4f006b420 0x7fc4f006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4f8088dd0 con 0x7fc504164920 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc5017fa700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] conn(0x7fc4f00735c0 0x7fc4f00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4f0076090 con 0x7fc4f00735c0 2024-01-16T14:54:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4f8053cb0 con 0x7fc504164920 2024-01-16T14:54:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7fc503fff700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] conn(0x7fc4f00735c0 0x7fc4f00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7f4105d9b700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] conn(0x7f40e8073640 0x7f40e8075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.027+0000 7f9fe7fff700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] conn(0x7f9fc8073640 0x7f9fc8075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7fc503fff700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] conn(0x7fc4f00735c0 0x7fc4f00759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7f1be3fff700 1 -- 172.21.15.26:0/4086321882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8004230 con 0x7f1bec15c010 2024-01-16T14:54:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7f1be3fff700 1 -- 172.21.15.26:0/4086321882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be8004390 con 0x7f1bec15c010 2024-01-16T14:54:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7f1be3fff700 1 -- 172.21.15.26:0/4086321882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be80036a0 con 0x7f1bec15c010 2024-01-16T14:54:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.028+0000 7fc508ee6700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc4f006b420 0x7fc4f006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== osd.0 v2:172.21.15.26:6808/1314027006 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9fc8076110 con 0x7f9fc8073640 2024-01-16T14:54:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f95c4cdb700 1 Processor -- start 2024-01-16T14:54:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7fc508ee6700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc4f006b420 0x7fc4f006d8e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc504069260 tx=0x7fc4fc009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 -- 172.21.15.26:0/4086321882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 msgr2=0x7f1bec0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 --2- 172.21.15.26:0/4086321882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec0cd2b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0b7d50 tx=0x7f1be8009bd0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 -- 172.21.15.26:0/4086321882 shutdown_connections 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 --2- 172.21.15.26:0/4086321882 >> v2:172.21.15.93:3300/0 conn(0x7f1bec0cf120 0x7f1bec06faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 --2- 172.21.15.26:0/4086321882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec0cd2b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 -- 172.21.15.26:0/4086321882 >> 172.21.15.26:0/4086321882 conn(0x7f1bec0b3790 msgr2=0x7f1bec0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 -- 172.21.15.26:0/4086321882 shutdown_connections 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f410659c700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40e806b420 0x7f40e806d8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4108068ce0 tx=0x7f410000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== osd.3 v2:172.21.15.26:6816/600555792 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc4f0076090 con 0x7fc4f00735c0 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f95c4cdb700 1 -- start start 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7f95c4cdb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95c0140270 0x7f95c015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7f95c4cdb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f95c0072350 0x7f95c00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7f95c4cdb700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c015e8f0 con 0x7f95c00b4730 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7f95c4cdb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c0143940 con 0x7f95c0140270 2024-01-16T14:54:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.030+0000 7f95c4cdb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c00b4c80 con 0x7f95c0072350 2024-01-16T14:54:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.029+0000 7f1bf397f700 1 -- 172.21.15.26:0/4086321882 wait complete. 2024-01-16T14:54:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95c0140270 0x7f95c015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95c0140270 0x7f95c015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49626/0 (socket says 172.21.15.26:49626) 2024-01-16T14:54:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 -- 172.21.15.26:0/2528008454 learned_addr learned my addr 172.21.15.26:0/2528008454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f1bf397f700 1 Processor -- start 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 -- 172.21.15.26:0/2528008454 >> v2:172.21.15.93:3300/0 conn(0x7f95c0072350 msgr2=0x7f95c00cde70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 --2- 172.21.15.26:0/2528008454 >> v2:172.21.15.93:3300/0 conn(0x7f95c0072350 0x7f95c00cde70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 -- 172.21.15.26:0/2528008454 >> v1:172.21.15.93:6789/0 conn(0x7f95c00b4730 legacy=0x7f95c0071c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 -- 172.21.15.26:0/2528008454 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00cd390 con 0x7f95c0140270 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f95bf7fe700 1 --2- 172.21.15.26:0/2528008454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0140270 0x7f95c015e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95c00ccb80 tx=0x7f95a800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9b99762e825fc7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1e68006d50 con 0x7f1e7006d400 2024-01-16T14:54:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e700ae1f0 con 0x7f1e7006d400 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f1e65ffb700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1e5406b420 0x7f1e5406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e68089800 con 0x7f1e7006d400 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f1e65ffb700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] conn(0x7f1e540735c0 0x7f1e540759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.033+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e54076090 con 0x7f1e540735c0 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f95be7fc700 1 -- 172.21.15.26:0/2528008454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8010070 con 0x7f95c0140270 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f95be7fc700 1 -- 172.21.15.26:0/2528008454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800da40 con 0x7f95c0140270 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.032+0000 7f95be7fc700 1 -- 172.21.15.26:0/2528008454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800ddd0 con 0x7f95c0140270 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.033+0000 7f95c4cdb700 1 -- 172.21.15.26:0/2528008454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0140270 msgr2=0x7f95c015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.033+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/2528008454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0140270 0x7f95c015e3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95c00ccb80 tx=0x7f95a800d330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.033+0000 7f1e75fbf700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] conn(0x7f1e540735c0 0x7f1e540759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1e74fbd700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1e5406b420 0x7f1e5406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.033+0000 7f1bf397f700 1 -- start start 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bec0ce560 0x7f1bec1f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bec0cf120 0x7f1bec0b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c8e00 con 0x7f1bec15c010 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1f2100 con 0x7f1bec0ce560 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf397f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1f23e0 con 0x7f1bec0cf120 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf1f1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf1f1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49642/0 (socket says 172.21.15.26:49642) 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf1f1c700 1 -- 172.21.15.26:0/2569210724 learned_addr learned my addr 172.21.15.26:0/2569210724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e68009890 con 0x7f1e7006d400 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf171b700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bec0ce560 0x7f1bec1f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1e74fbd700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1e5406b420 0x7f1e5406d8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004720 tx=0x7f1e60008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1e75fbf700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] conn(0x7f1e540735c0 0x7f1e540759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf1f1c700 1 -- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bec0cf120 msgr2=0x7f1bec0b7500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.034+0000 7f1bf1f1c700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bec0cf120 0x7f1bec0b7500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1bf1f1c700 1 -- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bec0ce560 msgr2=0x7f1bec1f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1bf1f1c700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bec0ce560 0x7f1bec1f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1bf1f1c700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be8009550 con 0x7f1bec15c010 2024-01-16T14:54:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f36250cb700 1 Processor -- start 2024-01-16T14:54:52.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1bf1f1c700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec159ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b970 tx=0x7f1be400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400c9b0 con 0x7f1bec15c010 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400cb30 con 0x7f1bec15c010 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4014440 con 0x7f1bec15c010 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f36250cb700 1 -- start start 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.035+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1f26c0 con 0x7f1bec15c010 2024-01-16T14:54:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec0b7a40 con 0x7f1bec15c010 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== osd.4 v2:172.21.15.93:6808/2738183985 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1e54076090 con 0x7f1e540735c0 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f36250cb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 0x7f36200d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f36250cb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f362006e2d0 0x7f362006e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f36250cb700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362006d880 con 0x7f362015f390 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f36250cb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362006deb0 con 0x7f362006e2d0 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.036+0000 7f36250cb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362006eca0 con 0x7f3620144670 2024-01-16T14:54:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 -- 172.21.15.26:0/2528008454 shutdown_connections 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/2528008454 >> v2:172.21.15.93:3300/0 conn(0x7f95c0072350 0x7f95c00cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/2528008454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0140270 0x7f95c015e3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 -- 172.21.15.26:0/2528008454 >> 172.21.15.26:0/2528008454 conn(0x7f95c00b1730 msgr2=0x7f95c00b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 -- 172.21.15.26:0/2528008454 shutdown_connections 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.037+0000 7f95c4cdb700 1 -- 172.21.15.26:0/2528008454 wait complete. 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.039+0000 7f95c4cdb700 1 Processor -- start 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.039+0000 7f95c4cdb700 1 -- start start 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 0x7f95c0139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95c00b4730 0x7f95c015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95c0140270 0x7f95c015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0158f60 con 0x7f95c0072350 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c015ce60 con 0x7f95c0140270 2024-01-16T14:54:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95c4cdb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c015c5c0 con 0x7f95c00b4730 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 0x7f95c0139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95bf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 0x7f95c0139e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49646/0 (socket says 172.21.15.26:49646) 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f95bf7fe700 1 -- 172.21.15.26:0/230211497 learned_addr learned my addr 172.21.15.26:0/230211497 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 -- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95c00b4730 msgr2=0x7f95c015b990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95c00b4730 0x7f95c015b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 -- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95c0140270 msgr2=0x7f95c015bed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95c0140270 0x7f95c015bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80085d0 con 0x7f95c0072350 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f95bf7fe700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 0x7f95c0139e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.031+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== osd.2 v2:172.21.15.26:6824/1003733076 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f40e8076110 con 0x7f40e8073640 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361f7fe700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f362015f390 0x7f36200b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49698/0 (socket says 172.21.15.26:49698) 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361f7fe700 1 -- 172.21.15.26:0/824429602 learned_addr learned my addr 172.21.15.26:0/824429602 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361ffff700 1 --2- 172.21.15.26:0/824429602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 0x7f36200d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361ffff700 1 -- 172.21.15.26:0/824429602 >> v2:172.21.15.26:3300/0 conn(0x7f362006e2d0 msgr2=0x7f362006e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361ffff700 1 --2- 172.21.15.26:0/824429602 >> v2:172.21.15.26:3300/0 conn(0x7f362006e2d0 0x7f362006e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361ffff700 1 -- 172.21.15.26:0/824429602 >> v1:172.21.15.93:6789/0 conn(0x7f362015f390 legacy=0x7f36200b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.041+0000 7f361ffff700 1 -- 172.21.15.26:0/824429602 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200ce760 con 0x7f3620144670 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.042+0000 7f361ffff700 1 --2- 172.21.15.26:0/824429602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 0x7f36200d0120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7d485d44412d8a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.043+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1be401d3b0 con 0x7f1bec15c010 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bec0aea70 con 0x7f1bec15c010 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1be1ffb700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bd006b420 0x7f1bd006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be4089990 con 0x7f1bec15c010 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1be1ffb700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] conn(0x7f1bd00735c0 0x7f1bd00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bd007a4a0 con 0x7f1bd00735c0 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.044+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be40547e0 con 0x7f1bec15c010 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8009b90 con 0x7f95c0072350 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8009cf0 con 0x7f95c0072350 2024-01-16T14:54:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80038a0 con 0x7f95c0072350 2024-01-16T14:54:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f1bf171b700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bd006b420 0x7f1bd006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f1bf171b700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bd006b420 0x7f1bd006d8e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c015c8a0 con 0x7f95c0072350 2024-01-16T14:54:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.045+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c006cc70 con 0x7f95c0072350 2024-01-16T14:54:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.046+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95c00cd040 con 0x7f95c0072350 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.047+0000 7f361effd700 1 -- 172.21.15.26:0/824429602 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800e070 con 0x7f3620144670 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.047+0000 7f361effd700 1 -- 172.21.15.26:0/824429602 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800bad0 con 0x7f3620144670 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.047+0000 7f361effd700 1 -- 172.21.15.26:0/824429602 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800be80 con 0x7f3620144670 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.048+0000 7f36250cb700 1 -- 172.21.15.26:0/824429602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 msgr2=0x7f36200d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.048+0000 7f36250cb700 1 --2- 172.21.15.26:0/824429602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 0x7f36200d0120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 -- 172.21.15.26:0/824429602 shutdown_connections 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 --2- 172.21.15.26:0/824429602 >> v2:172.21.15.26:3300/0 conn(0x7f362006e2d0 0x7f362006e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 --2- 172.21.15.26:0/824429602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3620144670 0x7f36200d0120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 -- 172.21.15.26:0/824429602 >> 172.21.15.26:0/824429602 conn(0x7f36200cd490 msgr2=0x7f36200cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 -- 172.21.15.26:0/824429602 shutdown_connections 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.050+0000 7f36250cb700 1 -- 172.21.15.26:0/824429602 wait complete. 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.052+0000 7f36250cb700 1 Processor -- start 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.053+0000 7f36250cb700 1 -- start start 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f362006e2d0 0x7f362016b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 0x7f36200ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f362015f390 0x7f36200adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b02c0 con 0x7f3620144670 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620174700 con 0x7f362015f390 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f36250cb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b1e50 con 0x7f362006e2d0 2024-01-16T14:54:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 0x7f36200ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 0x7f36200ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49658/0 (socket says 172.21.15.26:49658) 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f361f7fe700 1 -- 172.21.15.26:0/1529552060 learned_addr learned my addr 172.21.15.26:0/1529552060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.054+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f95a8003a00 con 0x7f95c0072350 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f95b7fff700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f95a006b420 0x7f95a006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95a808ba90 con 0x7f95c0072350 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f95beffd700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f95a006b420 0x7f95a006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f95b7fff700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] conn(0x7f95a0073640 0x7f95a0075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95a0076110 con 0x7f95a0073640 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a808be40 con 0x7f95c0072350 2024-01-16T14:54:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f95beffd700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f95a006b420 0x7f95a006d8e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95c015d4d0 tx=0x7f95b0006c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f36248ca700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f362015f390 0x7f36200adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 -- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f362006e2d0 msgr2=0x7f362016b460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f362006e2d0 0x7f362016b460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 -- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f362015f390 msgr2=0x7f36200adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f362015f390 0x7f36200adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009a70 con 0x7f3620144670 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7f361f7fe700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 0x7f36200ada10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f360800c780 tx=0x7f360800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608004c80 con 0x7f3620144670 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608004de0 con 0x7f3620144670 2024-01-16T14:54:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080072f0 con 0x7f3620144670 2024-01-16T14:54:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f1bf0f1a700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] conn(0x7f1bd00735c0 0x7f1bd00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7f95bffff700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] conn(0x7f95a0073640 0x7f95a0075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7f1bf0f1a700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] conn(0x7f1bd00735c0 0x7f1bd00759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36200b2130 con 0x7f3620144670 2024-01-16T14:54:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36201743b0 con 0x7f3620144670 2024-01-16T14:54:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7f95bffff700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] conn(0x7f95a0073640 0x7f95a0075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.058+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36200b4150 con 0x7f3620144670 2024-01-16T14:54:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.058+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== osd.6 v2:172.21.15.93:6800/1448583470 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1bd007a4a0 con 0x7f1bd00735c0 2024-01-16T14:54:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.059+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== osd.1 v2:172.21.15.26:6800/795239518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f95a0076110 con 0x7f95a0073640 2024-01-16T14:54:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.059+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 --> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb644002d40 con 0x7fb6500734f0 2024-01-16T14:54:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.060+0000 7fb6617fa700 1 -- 172.21.15.26:0/3683572429 <== osd.5 v2:172.21.15.93:6823/1651349923 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb644002d40 con 0x7fb6500734f0 2024-01-16T14:54:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.061+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3608007450 con 0x7f3620144670 2024-01-16T14:54:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 --> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc504068790 con 0x7fc4f00735c0 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.055+0000 7fc5017fa700 1 -- 172.21.15.26:0/435607634 <== osd.3 v2:172.21.15.26:6816/600555792 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc504068790 con 0x7fc4f00735c0 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] conn(0x7fc4f00735c0 msgr2=0x7fc4f00759e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.26:6816/600555792,v1:172.21.15.26:6817/600555792] conn(0x7fc4f00735c0 0x7fc4f00759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc4f006b420 msgr2=0x7fc4f006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc4f006b420 0x7fc4f006d8e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc504069260 tx=0x7fc4fc009000 comp rx=0 tx=0).stop 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 msgr2=0x7fc504164db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc50b14a700 1 --2- 172.21.15.26:0/435607634 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc504164920 0x7fc504164db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800c3c0 tx=0x7fc4f800c6d0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.056+0000 7fc5096e7700 1 -- 172.21.15.26:0/435607634 reap_dead start 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 shutdown_connections 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 >> 172.21.15.26:0/435607634 conn(0x7fc5040cf980 msgr2=0x7fc5040cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 shutdown_connections 2024-01-16T14:54:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.057+0000 7fc50b14a700 1 -- 172.21.15.26:0/435607634 wait complete. 2024-01-16T14:54:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.061+0000 7f361cff9700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f360006b420 0x7f360006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.061+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 --> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e70147ae0 con 0x7f1e540735c0 2024-01-16T14:54:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.061+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3608089a60 con 0x7f3620144670 2024-01-16T14:54:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f361ffff700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f360006b420 0x7f360006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] conn(0x7fb6500734f0 msgr2=0x7fb650075910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7fb66abfb700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6823/1651349923,v1:172.21.15.93:6825/1651349923] conn(0x7fb6500734f0 0x7fb650075910 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f361cff9700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] conn(0x7f3600073640 0x7f3600075a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3600076110 con 0x7f3600073640 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3608089e10 con 0x7f3620144670 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f36248ca700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] conn(0x7f3600073640 0x7f3600075a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f361ffff700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f360006b420 0x7f360006d8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3618000f80 tx=0x7f3618011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.062+0000 7f36248ca700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] conn(0x7f3600073640 0x7f3600075a60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb65006b350 msgr2=0x7fb65006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb65006b350 0x7fb65006d810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb66415d8d0 tx=0x7fb65c00d040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 msgr2=0x7fb6640b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 --2- 172.21.15.26:0/3683572429 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb664071390 0x7fb6640b29b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb65800d370 tx=0x7fb658004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb669198700 1 -- 172.21.15.26:0/3683572429 reap_dead start 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 shutdown_connections 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 >> 172.21.15.26:0/3683572429 conn(0x7fb6640cf980 msgr2=0x7fb6640b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 shutdown_connections 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7fb66abfb700 1 -- 172.21.15.26:0/3683572429 wait complete. 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.063+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== osd.7 v2:172.21.15.93:6816/3713418070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3600076110 con 0x7f3600073640 2024-01-16T14:54:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.065+0000 7f1e65ffb700 1 -- 172.21.15.26:0/2497122540 <== osd.4 v2:172.21.15.93:6808/2738183985 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e6c0095f0 con 0x7f1e540735c0 2024-01-16T14:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.066+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 --> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4108075480 con 0x7f40e8073640 2024-01-16T14:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.066+0000 7f40f6ffd700 1 -- 172.21.15.26:0/2540009275 <== osd.2 v2:172.21.15.26:6824/1003733076 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4108075480 con 0x7f40e8073640 2024-01-16T14:54:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.067+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] conn(0x7f40e8073640 msgr2=0x7f40e8075a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.067+0000 7f410c973700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:6824/1003733076,v1:172.21.15.26:6825/1003733076] conn(0x7f40e8073640 0x7f40e8075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40e806b420 msgr2=0x7f40e806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f410c973700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f40e806b420 0x7f40e806d8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4108068ce0 tx=0x7f410000b040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 msgr2=0x7f4108165d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f410c973700 1 --2- 172.21.15.26:0/2540009275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41080b3e10 0x7f4108165d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41080c9f80 tx=0x7f40f800e9e0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f4106d9d700 1 -- 172.21.15.26:0/2540009275 reap_dead start 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.067+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] conn(0x7f1e540735c0 msgr2=0x7f1e540759e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.067+0000 7f1e77a22700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6808/2738183985,v1:172.21.15.93:6809/2738183985] conn(0x7f1e540735c0 0x7f1e540759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1e5406b420 msgr2=0x7f1e5406d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f1e77a22700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1e5406b420 0x7f1e5406d8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004720 tx=0x7f1e60008030 comp rx=0 tx=0).stop 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7006d400 msgr2=0x7f1e700cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.068+0000 7f1e77a22700 1 --2- 172.21.15.26:0/2497122540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7006d400 0x7f1e700cd350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800be20 tx=0x7f1e68009f70 comp rx=0 tx=0).stop 2024-01-16T14:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 shutdown_connections 2024-01-16T14:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 >> 172.21.15.26:0/2540009275 conn(0x7f41080cf980 msgr2=0x7f410816d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f1e75fbf700 1 -- 172.21.15.26:0/2497122540 reap_dead start 2024-01-16T14:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 shutdown_connections 2024-01-16T14:54:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 >> 172.21.15.26:0/2497122540 conn(0x7f1e700cfa10 msgr2=0x7f1e700cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 shutdown_connections 2024-01-16T14:54:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.069+0000 7f410c973700 1 -- 172.21.15.26:0/2540009275 wait complete. 2024-01-16T14:54:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.071+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 --> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fe80b31b0 con 0x7f9fc8073640 2024-01-16T14:54:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.072+0000 7f9fe4ff9700 1 -- 172.21.15.26:0/4035208602 <== osd.0 v2:172.21.15.26:6808/1314027006 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9fe000a700 con 0x7f9fc8073640 2024-01-16T14:54:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.075+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] conn(0x7f9fc8073640 msgr2=0x7f9fc8075a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.075+0000 7f9fedc5a700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:6808/1314027006,v1:172.21.15.26:6809/1314027006] conn(0x7f9fc8073640 0x7f9fc8075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 shutdown_connections 2024-01-16T14:54:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f1e77a22700 1 -- 172.21.15.26:0/2497122540 wait complete. 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9fc806b420 msgr2=0x7f9fc806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9fc806b420 0x7f9fc806d8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fe816d4b0 tx=0x7f9fd8006c60 comp rx=0 tx=0).stop 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 msgr2=0x7f9fe806e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 --2- 172.21.15.26:0/4035208602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fe80d04a0 0x7f9fe806e580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006fd0 tx=0x7f9fd0004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fe7fff700 1 -- 172.21.15.26:0/4035208602 reap_dead start 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 shutdown_connections 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.078+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 >> 172.21.15.26:0/4035208602 conn(0x7f9fe80cf980 msgr2=0x7f9fe80cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.083+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 shutdown_connections 2024-01-16T14:54:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.083+0000 7f9fedc5a700 1 -- 172.21.15.26:0/4035208602 wait complete. 2024-01-16T14:54:52.103 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-01-16T14:54:52.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-16T14:54:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.085+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 --> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95c00cd950 con 0x7f95a0073640 2024-01-16T14:54:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.086+0000 7f95b7fff700 1 -- 172.21.15.26:0/230211497 <== osd.1 v2:172.21.15.26:6800/795239518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f95b800a720 con 0x7f95a0073640 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.087+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] conn(0x7f95a0073640 msgr2=0x7f95a0075a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.087+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:6800/795239518,v1:172.21.15.26:6801/795239518] conn(0x7f95a0073640 0x7f95a0075a60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f95a006b420 msgr2=0x7f95a006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f95a006b420 0x7f95a006d8e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f95c015d4d0 tx=0x7f95b0006c60 comp rx=0 tx=0).stop 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 msgr2=0x7f95c0139e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f95c4cdb700 1 --2- 172.21.15.26:0/230211497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95c0072350 0x7f95c0139e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000c00 tx=0x7f95a8004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f95bffff700 1 -- 172.21.15.26:0/230211497 reap_dead start 2024-01-16T14:54:52.106 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:52.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-16T14:54:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.090+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 --> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1bec06f2a0 con 0x7f1bd00735c0 2024-01-16T14:54:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 shutdown_connections 2024-01-16T14:54:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 >> 172.21.15.26:0/230211497 conn(0x7f95c00b1730 msgr2=0x7f95c00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1be1ffb700 1 -- 172.21.15.26:0/2569210724 <== osd.6 v2:172.21.15.93:6800/1448583470 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1bdc003e90 con 0x7f1bd00735c0 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] conn(0x7f1bd00735c0 msgr2=0x7f1bd00759e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6800/1448583470,v1:172.21.15.93:6801/1448583470] conn(0x7f1bd00735c0 0x7f1bd00759e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bd006b420 msgr2=0x7f1bd006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bd006b420 0x7f1bd006d8e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be8008030 comp rx=0 tx=0).stop 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 msgr2=0x7f1bec159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf397f700 1 --2- 172.21.15.26:0/2569210724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bec15c010 0x7f1bec159ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1be400b970 tx=0x7f1be400bd30 comp rx=0 tx=0).stop 2024-01-16T14:54:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.091+0000 7f1bf1f1c700 1 -- 172.21.15.26:0/2569210724 reap_dead start 2024-01-16T14:54:52.109 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:52.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-16T14:54:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.093+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 shutdown_connections 2024-01-16T14:54:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.093+0000 7f95c4cdb700 1 -- 172.21.15.26:0/230211497 wait complete. 2024-01-16T14:54:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.094+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 --> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36200b1fc0 con 0x7f3600073640 2024-01-16T14:54:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.094+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 shutdown_connections 2024-01-16T14:54:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.094+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 >> 172.21.15.26:0/2569210724 conn(0x7f1bec0b3790 msgr2=0x7f1bec0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.094+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 shutdown_connections 2024-01-16T14:54:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.094+0000 7f1bf397f700 1 -- 172.21.15.26:0/2569210724 wait complete. 2024-01-16T14:54:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f361cff9700 1 -- 172.21.15.26:0/1529552060 <== osd.7 v2:172.21.15.93:6816/3713418070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f36200b1fc0 con 0x7f3600073640 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] conn(0x7f3600073640 msgr2=0x7f3600075a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6816/3713418070,v1:172.21.15.93:6817/3713418070] conn(0x7f3600073640 0x7f3600075a60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f360006b420 msgr2=0x7f360006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f360006b420 0x7f360006d8e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3618000f80 tx=0x7f3618011000 comp rx=0 tx=0).stop 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 msgr2=0x7f36200ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36250cb700 1 --2- 172.21.15.26:0/1529552060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3620144670 0x7f36200ada10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f360800c780 tx=0x7f360800cb40 comp rx=0 tx=0).stop 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.095+0000 7f36248ca700 1 -- 172.21.15.26:0/1529552060 reap_dead start 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.097+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 shutdown_connections 2024-01-16T14:54:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.097+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 >> 172.21.15.26:0/1529552060 conn(0x7f36200cd490 msgr2=0x7f3620134b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.097+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 shutdown_connections 2024-01-16T14:54:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.097+0000 7f36250cb700 1 -- 172.21.15.26:0/1529552060 wait complete. 2024-01-16T14:54:52.113 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-16T14:54:52.114 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-16T14:54:52.115 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-16T14:54:52.116 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-01-16T14:54:52.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-16T14:54:52.117 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:52.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.390+0000 7f32948ef700 1 Processor -- start 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.390+0000 7f32948ef700 1 -- start start 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.391+0000 7f32948ef700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3290070a80 0x7f3290070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.391+0000 7f32948ef700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3290166600 0x7f32900cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.391+0000 7f32948ef700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32900ca8e0 con 0x7f3290145870 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.391+0000 7f32948ef700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3290166a10 con 0x7f3290166600 2024-01-16T14:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.391+0000 7f32948ef700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32900713a0 con 0x7f3290070a80 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.392+0000 7f328dd9b700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f3290145870 0x7f3290165ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49708/0 (socket says 172.21.15.26:49708) 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.392+0000 7f328dd9b700 1 -- 172.21.15.26:0/913418317 learned_addr learned my addr 172.21.15.26:0/913418317 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.392+0000 7f328e59c700 1 --2- 172.21.15.26:0/913418317 >> v2:172.21.15.93:3300/0 conn(0x7f3290070a80 0x7f3290070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.392+0000 7f328d59a700 1 -- 172.21.15.26:0/913418317 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604115125 0 0) 0x7f32900ca8e0 con 0x7f3290145870 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.392+0000 7f328d59a700 1 -- 172.21.15.26:0/913418317 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32780035b0 con 0x7f3290145870 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328e59c700 1 -- 172.21.15.26:0/913418317 >> v2:172.21.15.26:3300/0 conn(0x7f3290166600 msgr2=0x7f32900cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328e59c700 1 --2- 172.21.15.26:0/913418317 >> v2:172.21.15.26:3300/0 conn(0x7f3290166600 0x7f32900cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328e59c700 1 -- 172.21.15.26:0/913418317 >> v1:172.21.15.93:6789/0 conn(0x7f3290145870 legacy=0x7f3290165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328e59c700 1 -- 172.21.15.26:0/913418317 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900c9fd0 con 0x7f3290070a80 2024-01-16T14:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328e59c700 1 --2- 172.21.15.26:0/913418317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290070e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32900731e0 tx=0x7f328800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778f27e750cec24d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328d59a700 1 -- 172.21.15.26:0/913418317 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328800e070 con 0x7f3290070a80 2024-01-16T14:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328d59a700 1 -- 172.21.15.26:0/913418317 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328800bb20 con 0x7f3290070a80 2024-01-16T14:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.393+0000 7f328d59a700 1 -- 172.21.15.26:0/913418317 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328800be90 con 0x7f3290070a80 2024-01-16T14:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.394+0000 7f32948ef700 1 -- 172.21.15.26:0/913418317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 msgr2=0x7f3290070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.394+0000 7f32948ef700 1 --2- 172.21.15.26:0/913418317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290070e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32900731e0 tx=0x7f328800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 -- 172.21.15.26:0/913418317 shutdown_connections 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 --2- 172.21.15.26:0/913418317 >> v2:172.21.15.26:3300/0 conn(0x7f3290166600 0x7f32900cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 --2- 172.21.15.26:0/913418317 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290070e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 -- 172.21.15.26:0/913418317 >> 172.21.15.26:0/913418317 conn(0x7f32900cf980 msgr2=0x7f32900cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 -- 172.21.15.26:0/913418317 shutdown_connections 2024-01-16T14:54:52.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.395+0000 7f32948ef700 1 -- 172.21.15.26:0/913418317 wait complete. 2024-01-16T14:54:52.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.398+0000 7f32948ef700 1 Processor -- start 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.398+0000 7f32948ef700 1 -- start start 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 0x7f329015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 0x7f329015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329015f1d0 con 0x7f3290166600 2024-01-16T14:54:52.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329015e370 con 0x7f3290070a80 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.399+0000 7f32948ef700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329015e5c0 con 0x7f3290145870 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 0x7f329015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 0x7f329015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49674/0 (socket says 172.21.15.26:49674) 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328ed9d700 1 -- 172.21.15.26:0/2476391434 learned_addr learned my addr 172.21.15.26:0/2476391434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 0x7f329015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 -- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 msgr2=0x7f3290144bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290144bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 -- 172.21.15.26:0/2476391434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 msgr2=0x7f329015a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 0x7f329015a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.400+0000 7f328dd9b700 1 -- 172.21.15.26:0/2476391434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288009a70 con 0x7f3290145870 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.401+0000 7f328dd9b700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 0x7f329015de30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f328000c490 tx=0x7f328000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.401+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280005e30 con 0x7f3290145870 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.401+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329015e8a0 con 0x7f3290145870 2024-01-16T14:54:52.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.401+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329015ee60 con 0x7f3290145870 2024-01-16T14:54:52.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.402+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32800070d0 con 0x7f3290145870 2024-01-16T14:54:52.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.402+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280007440 con 0x7f3290145870 2024-01-16T14:54:52.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.403+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32700052f0 con 0x7f3290145870 2024-01-16T14:54:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.404+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f32800075a0 con 0x7f3290145870 2024-01-16T14:54:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.405+0000 7f327effd700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 0x7f327806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.405+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3280088ea0 con 0x7f3290145870 2024-01-16T14:54:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.404+0000 7fed3f22c700 1 Processor -- start 2024-01-16T14:54:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7f328e59c700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 0x7f327806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7f328e59c700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 0x7f327806dc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3288014040 tx=0x7f3288011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.405+0000 7fed3f22c700 1 -- start start 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3f22c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed3814bf10 0x7fed38070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3f22c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fed380cbb80 0x7fed3806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3f22c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed380cc080 con 0x7fed38165f70 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3f22c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed380cdf60 con 0x7fed380cbb80 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3f22c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3806e950 con 0x7fed3814bf10 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed37fff700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fed380cbb80 0x7fed3806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed37fff700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fed380cbb80 0x7fed3806cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2024-01-16T14:54:52.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed37fff700 1 -- 172.21.15.26:0/1082928572 learned_addr learned my addr 172.21.15.26:0/1082928572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fea70312700 1 Processor -- start 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fea70312700 1 -- start start 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.406+0000 7fed3cfc8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed3814bf10 0x7fed38070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fed37fff700 1 -- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed3814bf10 msgr2=0x7fed38070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fed37fff700 1 --2- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed3814bf10 0x7fed38070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fed37fff700 1 -- 172.21.15.26:0/1082928572 >> v1:172.21.15.26:6789/0 conn(0x7fed38165f70 legacy=0x7fed380ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fed37fff700 1 -- 172.21.15.26:0/1082928572 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3806ec30 con 0x7fed380cbb80 2024-01-16T14:54:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fed37fff700 1 --2- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed380cbb80 0x7fed3806cae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098f0 tx=0x7fed2c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c61cacfdf0ec78b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fea70312700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fea6806cae0 0x7fea6816dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fea70312700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea680cab10 con 0x7fea68071190 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fea70312700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea680ca1e0 con 0x7fea680ce440 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.407+0000 7fea70312700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea680c9fd0 con 0x7fea6806cae0 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.408+0000 7fea6e8af700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fea68071190 0x7fea680cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58396/0 (socket says 172.21.15.26:58396) 2024-01-16T14:54:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.408+0000 7fea6e8af700 1 -- 172.21.15.26:0/1903322263 learned_addr learned my addr 172.21.15.26:0/1903322263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fed377fe700 1 -- 172.21.15.26:0/1082928572 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c004230 con 0x7fed380cbb80 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fed377fe700 1 -- 172.21.15.26:0/1082928572 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c004390 con 0x7fed380cbb80 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fed377fe700 1 -- 172.21.15.26:0/1082928572 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0036a0 con 0x7fed380cbb80 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6e0ae700 1 --2- 172.21.15.26:0/1903322263 >> v2:172.21.15.93:3300/0 conn(0x7fea6806cae0 0x7fea6816dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6e0ae700 1 -- 172.21.15.26:0/1903322263 >> v1:172.21.15.93:6789/0 conn(0x7fea680ce440 legacy=0x7fea68070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6e0ae700 1 -- 172.21.15.26:0/1903322263 >> v1:172.21.15.26:6789/0 conn(0x7fea68071190 legacy=0x7fea680cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6e0ae700 1 -- 172.21.15.26:0/1903322263 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea680ca4c0 con 0x7fea6806cae0 2024-01-16T14:54:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7f281a872700 1 Processor -- start 2024-01-16T14:54:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7f281a872700 1 -- start start 2024-01-16T14:54:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6e0ae700 1 --2- 172.21.15.26:0/1903322263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea6806cae0 0x7fea6816dfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea64009840 tx=0x7fea64009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f091b8cb9bb35639 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.409+0000 7fea6d0ac700 1 -- 172.21.15.26:0/1903322263 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64004230 con 0x7fea6806cae0 2024-01-16T14:54:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.410+0000 7fea6d0ac700 1 -- 172.21.15.26:0/1903322263 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64004390 con 0x7fea6806cae0 2024-01-16T14:54:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.410+0000 7f489709b700 1 Processor -- start 2024-01-16T14:54:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.410+0000 7fed3f22c700 1 -- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed380cbb80 msgr2=0x7fed3806cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.410+0000 7fed3f22c700 1 --2- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed380cbb80 0x7fed3806cae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098f0 tx=0x7fed2c009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f489709b700 1 -- start start 2024-01-16T14:54:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fea70312700 1 -- 172.21.15.26:0/1903322263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea6806cae0 msgr2=0x7fea6816dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fea70312700 1 --2- 172.21.15.26:0/1903322263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea6806cae0 0x7fea6816dfb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea64009840 tx=0x7fea64009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fed3f22c700 1 -- 172.21.15.26:0/1082928572 shutdown_connections 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fed3f22c700 1 --2- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed3814bf10 0x7fed38070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fed3f22c700 1 --2- 172.21.15.26:0/1082928572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed380cbb80 0x7fed3806cae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fed3f22c700 1 -- 172.21.15.26:0/1082928572 >> 172.21.15.26:0/1082928572 conn(0x7fed380cf980 msgr2=0x7fed380cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3280053db0 con 0x7f3290145870 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f281a872700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2814161be0 0x7f281415f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f281a872700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281415f560 con 0x7f28140cb1d0 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f281a872700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281406d3d0 con 0x7f28141446a0 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f281a872700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2814162070 con 0x7f2814161be0 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f2813fff700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f28141446a0 0x7f28140caac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49722/0 (socket says 172.21.15.26:49722) 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7f2813fff700 1 -- 172.21.15.26:0/1814981575 learned_addr learned my addr 172.21.15.26:0/1814981575 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fea70312700 1 -- 172.21.15.26:0/1903322263 shutdown_connections 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fea70312700 1 --2- 172.21.15.26:0/1903322263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea6806cae0 0x7fea6816dfb0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea64009840 tx=0x7fea64009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.411+0000 7fea70312700 1 -- 172.21.15.26:0/1903322263 >> 172.21.15.26:0/1903322263 conn(0x7fea680cf980 msgr2=0x7fea680cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7fed3f22c700 1 -- 172.21.15.26:0/1082928572 shutdown_connections 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f489709b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4890071bb0 0x7f48900cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f489709b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f489015e3b0 0x7f4890071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f489709b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4890070580 con 0x7f48900d1650 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f489709b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4890070190 con 0x7f489015e3b0 2024-01-16T14:54:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f489709b700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489006feb0 con 0x7f4890071bb0 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f28137fe700 1 --2- 172.21.15.26:0/1814981575 >> v2:172.21.15.93:3300/0 conn(0x7f2814161be0 0x7f281415f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f28137fe700 1 -- 172.21.15.26:0/1814981575 >> v1:172.21.15.93:6789/0 conn(0x7f28141446a0 legacy=0x7f28140caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f28137fe700 1 -- 172.21.15.26:0/1814981575 >> v1:172.21.15.26:6789/0 conn(0x7f28140cb1d0 legacy=0x7f281406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7f28137fe700 1 -- 172.21.15.26:0/1814981575 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814162350 con 0x7f2814161be0 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.412+0000 7fed3f22c700 1 -- 172.21.15.26:0/1082928572 wait complete. 2024-01-16T14:54:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f4894e37700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4890071bb0 0x7f48900cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f488ffff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f489015e3b0 0x7f4890071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f488ffff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f489015e3b0 0x7f4890071f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49682/0 (socket says 172.21.15.26:49682) 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f488ffff700 1 -- 172.21.15.26:0/3516302615 learned_addr learned my addr 172.21.15.26:0/3516302615 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7fed3f22c700 1 Processor -- start 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7fed3f22c700 1 -- start start 2024-01-16T14:54:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f488ffff700 1 -- 172.21.15.26:0/3516302615 >> v2:172.21.15.93:3300/0 conn(0x7f4890071bb0 msgr2=0x7f48900cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f488ffff700 1 --2- 172.21.15.26:0/3516302615 >> v2:172.21.15.93:3300/0 conn(0x7f4890071bb0 0x7f48900cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f488ffff700 1 -- 172.21.15.26:0/3516302615 >> v1:172.21.15.93:6789/0 conn(0x7f48900d1650 legacy=0x7f489015eaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:52.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f488ffff700 1 -- 172.21.15.26:0/3516302615 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900b5a40 con 0x7f489015e3b0 2024-01-16T14:54:52.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f488ffff700 1 --2- 172.21.15.26:0/3516302615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f4890071f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4890074970 tx=0x7f488800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fa16047391623e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7f28137fe700 1 --2- 172.21.15.26:0/1814981575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814161be0 0x7f281415f020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f280c009d90 tx=0x7f280c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2049ba150f892d7d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed380cbb80 0x7fed38164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 0x7fed3815f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38160210 con 0x7fed380cbb80 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3815fc50 con 0x7fed3814bf10 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3f22c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3815ff30 con 0x7fed38165f70 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3d7c9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3d7c9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38494/0 (socket says 172.21.15.26:38494) 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7fed3d7c9700 1 -- 172.21.15.26:0/4255867490 learned_addr learned my addr 172.21.15.26:0/4255867490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7f488f7fe700 1 -- 172.21.15.26:0/3516302615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488800e070 con 0x7f489015e3b0 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7f488f7fe700 1 -- 172.21.15.26:0/3516302615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888003a20 con 0x7f489015e3b0 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7f488f7fe700 1 -- 172.21.15.26:0/3516302615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888003db0 con 0x7f489015e3b0 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f6e1566d700 1 Processor -- start 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.413+0000 7f6e1566d700 1 -- start start 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f6e1566d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e1006cae0 0x7f6e100ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f6e1566d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e100ce980 0x7f6e100cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f6e1566d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e100cab10 con 0x7f6e10070a80 2024-01-16T14:54:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f6e1566d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e100c9fd0 con 0x7f6e1006cae0 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.414+0000 7f6e1566d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100cd7e0 con 0x7f6e100ce980 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e100ce980 0x7f6e100cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e100ce980 0x7f6e100cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38492/0 (socket says 172.21.15.26:38492) 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7f6e0f7fe700 1 -- 172.21.15.26:0/1540217489 learned_addr learned my addr 172.21.15.26:0/1540217489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed37fff700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 0x7fed3815f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.415+0000 7f6e0e7fc700 1 --2- 172.21.15.26:0/1540217489 >> v2:172.21.15.93:3300/0 conn(0x7f6e1006cae0 0x7f6e100ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 -- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 msgr2=0x7fed3815f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 0x7fed3815f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 -- 172.21.15.26:0/4255867490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed380cbb80 msgr2=0x7fed38164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed380cbb80 0x7fed38164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 -- 172.21.15.26:0/4255867490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009550 con 0x7fed38165f70 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed37fff700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 0x7fed3815f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.416+0000 7fed3d7c9700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed3000b9e0 tx=0x7fed3000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7fea70312700 1 -- 172.21.15.26:0/1903322263 shutdown_connections 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f489709b700 1 -- 172.21.15.26:0/3516302615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 msgr2=0x7f4890071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f489709b700 1 --2- 172.21.15.26:0/3516302615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f4890071f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4890074970 tx=0x7f488800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f2812ffd700 1 -- 172.21.15.26:0/1814981575 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c00e070 con 0x7f2814161be0 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f2812ffd700 1 -- 172.21.15.26:0/1814981575 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c00bbd0 con 0x7f2814161be0 2024-01-16T14:54:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f2812ffd700 1 -- 172.21.15.26:0/1814981575 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c012430 con 0x7f2814161be0 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912412666 0 0) 0x7f6e100cab10 con 0x7f6e10070a80 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df80035b0 con 0x7f6e10070a80 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052747568 0 0) 0x7f6df80035b0 con 0x7f6e10070a80 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e100ce980 msgr2=0x7f6e100cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e100ce980 0x7f6e100cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 >> v2:172.21.15.93:3300/0 conn(0x7f6e1006cae0 msgr2=0x7f6e100ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 --2- 172.21.15.26:0/1540217489 >> v2:172.21.15.93:3300/0 conn(0x7f6e1006cae0 0x7f6e100ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100ca1e0 con 0x7f6e10070a80 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f6e00001670 con 0x7f6e10070a80 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 >> v1:172.21.15.26:6789/0 conn(0x7f6e10070a80 legacy=0x7f6e10070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df8003a00 0x7f6df8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df80043f0 0x7f6df80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df8004e10 0x7f6df8005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100c9fd0 con 0x7f6df80043f0 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e100cd7e0 con 0x7f6df8004e10 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80147a0 con 0x7f6df8003a00 2024-01-16T14:54:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fea70312700 1 -- 172.21.15.26:0/1903322263 wait complete. 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.417+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df8003a00 0x7f6df8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 -- 172.21.15.26:0/1814981575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814161be0 msgr2=0x7f281415f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 --2- 172.21.15.26:0/1814981575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814161be0 0x7f281415f020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f280c009d90 tx=0x7f280c00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 -- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df8004e10 msgr2=0x7f6df8005290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df8004e10 0x7f6df8005290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 -- 172.21.15.26:0/1540217489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df80043f0 msgr2=0x7f6df80048d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:52.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df80043f0 0x7f6df80048d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 -- 172.21.15.26:0/1540217489 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100ca1e0 con 0x7f6df8003a00 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df8003a00 0x7f6df8003e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c2c0 tx=0x7f6df8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d706a8aa103f2c31 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3000c950 con 0x7fed38165f70 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3000cab0 con 0x7fed38165f70 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed38160540 con 0x7fed38165f70 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3000ce60 con 0x7fed38165f70 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed381654b0 con 0x7fed38165f70 2024-01-16T14:54:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f489709b700 1 -- 172.21.15.26:0/3516302615 shutdown_connections 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f489709b700 1 --2- 172.21.15.26:0/3516302615 >> v2:172.21.15.93:3300/0 conn(0x7f4890071bb0 0x7f48900cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f489709b700 1 --2- 172.21.15.26:0/3516302615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f4890071f90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f489709b700 1 -- 172.21.15.26:0/3516302615 >> 172.21.15.26:0/3516302615 conn(0x7f48900d0b30 msgr2=0x7f48900d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 -- 172.21.15.26:0/1814981575 shutdown_connections 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 --2- 172.21.15.26:0/1814981575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814161be0 0x7f281415f020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 -- 172.21.15.26:0/1814981575 >> 172.21.15.26:0/1814981575 conn(0x7f28140cf980 msgr2=0x7f28140cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 -- 172.21.15.26:0/1814981575 shutdown_connections 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.419+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000cd80 con 0x7f6df8003a00 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.419+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e00012420 con 0x7f6df8003a00 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.419+0000 7f6e0dffb700 1 -- 172.21.15.26:0/1540217489 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e000127d0 con 0x7f6df8003a00 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.419+0000 7f6e1566d700 1 -- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df8003a00 msgr2=0x7f6df8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.419+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1540217489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df8003a00 0x7f6df8003e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c2c0 tx=0x7f6df8005880 comp rx=0 tx=0).stop 2024-01-16T14:54:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.420+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fed3001d3b0 con 0x7fed38165f70 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.418+0000 7f281a872700 1 -- 172.21.15.26:0/1814981575 wait complete. 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.420+0000 7f281a872700 1 Processor -- start 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.421+0000 7fed357fa700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 0x7fed2406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.421+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed300898b0 con 0x7fed38165f70 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.421+0000 7fed3cfc8700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 0x7fed2406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7fed3cfc8700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 0x7fed2406d810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed28006b20 tx=0x7fed28006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.421+0000 7f281a872700 1 -- start start 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28140cb1d0 0x7f28141413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 0x7f281415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814141c90 con 0x7f28140cb1d0 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814141f70 con 0x7f2814143ae0 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f281a872700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2814142250 con 0x7f28141446a0 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f28137fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f28137fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38510/0 (socket says 172.21.15.26:38510) 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f28137fe700 1 -- 172.21.15.26:0/3053386807 learned_addr learned my addr 172.21.15.26:0/3053386807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.422+0000 7f2813fff700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28140cb1d0 0x7f28141413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f489709b700 1 -- 172.21.15.26:0/3516302615 shutdown_connections 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f2818e0f700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 0x7f281415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 -- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 msgr2=0x7f281415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 0x7f281415d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 -- 172.21.15.26:0/3053386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28140cb1d0 msgr2=0x7f28141413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28140cb1d0 0x7f28141413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 -- 172.21.15.26:0/3053386807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c009a70 con 0x7f28141446a0 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f2818e0f700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 0x7f281415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.420+0000 7fea70312700 1 Processor -- start 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.421+0000 7fea70312700 1 -- start start 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea68071190 0x7fea680b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea680b2740 0x7fea680b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680b3110 con 0x7fea680ce440 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680b3350 con 0x7fea68071190 2024-01-16T14:54:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fea70312700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea680b3630 con 0x7fea680b2740 2024-01-16T14:54:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f6e0f7fe700 1 -- 172.21.15.26:0/1540217489 reap_dead start 2024-01-16T14:54:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f6e1566d700 1 -- 172.21.15.26:0/1540217489 shutdown_connections 2024-01-16T14:54:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f6e1566d700 1 -- 172.21.15.26:0/1540217489 >> 172.21.15.26:0/1540217489 conn(0x7f6e100cf980 msgr2=0x7f6e100cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f28137fe700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f280c014040 tx=0x7f280c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed381603a0 con 0x7fed38165f70 2024-01-16T14:54:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.424+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0048b0 con 0x7f28141446a0 2024-01-16T14:54:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.424+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c020070 con 0x7f28141446a0 2024-01-16T14:54:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.424+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0276c0 con 0x7f28141446a0 2024-01-16T14:54:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.423+0000 7f489709b700 1 -- 172.21.15.26:0/3516302615 wait complete. 2024-01-16T14:54:52.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.425+0000 7f489709b700 1 Processor -- start 2024-01-16T14:54:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.425+0000 7f6e1566d700 1 -- 172.21.15.26:0/1540217489 shutdown_connections 2024-01-16T14:54:52.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.425+0000 7f489709b700 1 -- start start 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4890071bb0 0x7f489014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48900d1650 0x7f48901f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489014c310 con 0x7f489015e3b0 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f8fc0 con 0x7f4890071bb0 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f489709b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48901f92a0 con 0x7f48900d1650 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f4895638700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f4895638700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49708/0 (socket says 172.21.15.26:49708) 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f4895638700 1 -- 172.21.15.26:0/542513902 learned_addr learned my addr 172.21.15.26:0/542513902 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 -- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48900d1650 msgr2=0x7f48901f8630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48900d1650 0x7f48901f8630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 -- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4890071bb0 msgr2=0x7f489014bdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4890071bb0 0x7f489014bdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 -- 172.21.15.26:0/542513902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4888009b30 con 0x7f489015e3b0 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f4895638700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f488000b3e0 tx=0x7f488000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.425+0000 7f6e1566d700 1 -- 172.21.15.26:0/1540217489 wait complete. 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.428+0000 7f6e1566d700 1 Processor -- start 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2814142530 con 0x7f28141446a0 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.426+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2814142af0 con 0x7f28141446a0 2024-01-16T14:54:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.428+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814143ec0 con 0x7f28141446a0 2024-01-16T14:54:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.428+0000 7f6e1566d700 1 -- start start 2024-01-16T14:54:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49720/0 (socket says 172.21.15.26:49720) 2024-01-16T14:54:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 -- 172.21.15.26:0/518537493 learned_addr learned my addr 172.21.15.26:0/518537493 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6e0ae700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea68071190 0x7fea680b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011830 con 0x7f489015e3b0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48800119b0 con 0x7f489015e3b0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011d60 con 0x7f489015e3b0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48901f9580 con 0x7f489015e3b0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.427+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489015ac40 con 0x7f489015e3b0 2024-01-16T14:54:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48700052f0 con 0x7f489015e3b0 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e1015b2c0 0x7f6e100b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 0x7f6e100cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 0x7f6e100cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e10201990 con 0x7f6e10200810 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1016a330 con 0x7f6e1015b2c0 2024-01-16T14:54:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7f6e1566d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e10200dd0 con 0x7f6e1015b6a0 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 -- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea680b2740 msgr2=0x7fea680b2bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea680b2740 0x7fea680b2bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 -- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea68071190 msgr2=0x7fea680b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea68071190 0x7fea680b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fea6d8ad700 1 -- 172.21.15.26:0/518537493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640094a0 con 0x7fea680ce440 2024-01-16T14:54:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.429+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed381603a0 con 0x7fed38165f70 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 0x7f6e100cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7f6e0f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 0x7f6e100cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49724/0 (socket says 172.21.15.26:49724) 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7f6e0f7fe700 1 -- 172.21.15.26:0/1582765878 learned_addr learned my addr 172.21.15.26:0/1582765878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e1015b2c0 0x7f6e100b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 0x7f6e100cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 -- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e1015b2c0 msgr2=0x7f6e100b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e1015b2c0 0x7f6e100b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 -- 172.21.15.26:0/1582765878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 msgr2=0x7f6e100cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 0x7f6e100cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f6e0e7fc700 1 -- 172.21.15.26:0/1582765878 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e102010b0 con 0x7f6e1015b6a0 2024-01-16T14:54:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f280c012430 con 0x7f28141446a0 2024-01-16T14:54:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f2810ff9700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 0x7f280006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.431+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f280c022070 con 0x7f28141446a0 2024-01-16T14:54:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.430+0000 7fea6d8ad700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea5800b4a0 tx=0x7fea5800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.432+0000 7f6e0e7fc700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 0x7f6e100cafa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e1006a6a0 tx=0x7f6e0400dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.432+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f488001a400 con 0x7f489015e3b0 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.432+0000 7f488d7fa700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 0x7f487c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.432+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f488008a100 con 0x7f489015e3b0 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.433+0000 7f4894e37700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 0x7f487c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.433+0000 7f4894e37700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 0x7f487c06d8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f488400a810 tx=0x7f4884008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.434+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f280c056fc0 con 0x7f28141446a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.434+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04004220 con 0x7f6e1015b6a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.434+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04004380 con 0x7f6e1015b6a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.434+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e040036a0 con 0x7f6e1015b6a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.435+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e10201390 con 0x7f6e1015b6a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.435+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e100b54e0 con 0x7f6e1015b6a0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.436+0000 7f2813fff700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 0x7f280006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.437+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e10075480 con 0x7f6e1015b6a0 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.437+0000 7f2813fff700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 0x7f280006d8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2804000f60 tx=0x7f2804007eb0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.437+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6e04003800 con 0x7f6e1015b6a0 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.437+0000 7f6df77fe700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 0x7f6dfc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.438+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e04014070 con 0x7f6e1015b6a0 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.438+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 0x7f6dfc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.438+0000 7f6e0effd700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 0x7f6dfc06d840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c1e0 tx=0x7f6e00011000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.439+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4880055090 con 0x7f489015e3b0 2024-01-16T14:54:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.439+0000 7f8467d61700 1 Processor -- start 2024-01-16T14:54:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58011830 con 0x7fea680ce440 2024-01-16T14:54:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7f8467d61700 1 -- start start 2024-01-16T14:54:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58011990 con 0x7fea680ce440 2024-01-16T14:54:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58011d40 con 0x7fea680ce440 2024-01-16T14:54:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea680b3910 con 0x7fea680ce440 2024-01-16T14:54:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.440+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea68154cf0 con 0x7fea680ce440 2024-01-16T14:54:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea500052f0 con 0x7fea680ce440 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f8467d61700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f846015bf10 0x7f8460066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f8467d61700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f84600af980 0x7f84600b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f8467d61700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84600b10c0 con 0x7f84600b1a90 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f8467d61700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84600af730 con 0x7f84600af980 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f8467d61700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84600b5ee0 con 0x7f846015bf10 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f84662fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f84600b1a90 0x7f846006b7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58426/0 (socket says 172.21.15.26:58426) 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.441+0000 7f84662fe700 1 -- 172.21.15.26:0/2512848602 learned_addr learned my addr 172.21.15.26:0/2512848602 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.442+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e040548c0 con 0x7f6e1015b6a0 2024-01-16T14:54:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.443+0000 7f8465afd700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f846015bf10 0x7f8460066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 --2- 172.21.15.26:0/2512848602 >> v2:172.21.15.93:3300/0 conn(0x7f84600af980 0x7f84600b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 -- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f846015bf10 msgr2=0x7f8460066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f846015bf10 0x7f8460066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 -- 172.21.15.26:0/2512848602 >> v1:172.21.15.26:6789/0 conn(0x7f84600b1a90 legacy=0x7f846006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 -- 172.21.15.26:0/2512848602 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600b61c0 con 0x7f84600af980 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f84652fc700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84600af980 0x7f84600b1550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8450009840 tx=0x7f8450009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d485fbed7ea8c5a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.444+0000 7f8464afb700 1 -- 172.21.15.26:0/2512848602 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450011830 con 0x7f84600af980 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.445+0000 7f8464afb700 1 -- 172.21.15.26:0/2512848602 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450011990 con 0x7f84600af980 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.445+0000 7f8464afb700 1 -- 172.21.15.26:0/2512848602 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450011d20 con 0x7f84600af980 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.446+0000 7f8467d61700 1 -- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84600af980 msgr2=0x7f84600b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.446+0000 7f8467d61700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84600af980 0x7f84600b1550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8450009840 tx=0x7f8450009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.447+0000 7f8467d61700 1 -- 172.21.15.26:0/2512848602 shutdown_connections 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.447+0000 7f8467d61700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f846015bf10 0x7f8460066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.447+0000 7f8467d61700 1 --2- 172.21.15.26:0/2512848602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84600af980 0x7f84600b1550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.447+0000 7f8467d61700 1 -- 172.21.15.26:0/2512848602 >> 172.21.15.26:0/2512848602 conn(0x7f84600b5070 msgr2=0x7f84600b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.450+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fea5801a400 con 0x7fea680ce440 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.451+0000 7fea5effd700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 0x7fea4c06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.451+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea5808a0e0 con 0x7fea680ce440 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.452+0000 7fea6e0ae700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 0x7fea4c06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.452+0000 7f8467d61700 1 -- 172.21.15.26:0/2512848602 shutdown_connections 2024-01-16T14:54:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.453+0000 7f27fdc2f700 1 Processor -- start 2024-01-16T14:54:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.453+0000 7f8467d61700 1 -- 172.21.15.26:0/2512848602 wait complete. 2024-01-16T14:54:52.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:54:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.455+0000 7f8467d61700 1 Processor -- start 2024-01-16T14:54:52.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:54:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.455+0000 7f8467d61700 1 -- start start 2024-01-16T14:54:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84600af980 0x7f8460159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8460145f40 0x7f846009d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8460155e70 con 0x7f84600af980 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846009c700 con 0x7f8460159870 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f8467d61700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846009c9e0 con 0x7f8460145f40 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.454+0000 7f27fdc2f700 1 -- start start 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f27fdc2f700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27f80722c0 0x7f27f80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f27fdc2f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f81440f0 0x7f27f806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f27fdc2f700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f8143ec0 con 0x7f27f8143ae0 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f27fdc2f700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80b40a0 con 0x7f27f80722c0 2024-01-16T14:54:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.456+0000 7f27fdc2f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80b3ae0 con 0x7f27f81440f0 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.458+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea58054fc0 con 0x7fea680ce440 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84662fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8460145f40 0x7f846009d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.458+0000 7f27f7fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f81440f0 0x7f27f806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f27f77fe700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f27f8143ae0 0x7f27f8071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49752/0 (socket says 172.21.15.26:49752) 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f27f77fe700 1 -- 172.21.15.26:0/2953090907 learned_addr learned my addr 172.21.15.26:0/2953090907 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47770/0 (socket says 172.21.15.26:47770) 2024-01-16T14:54:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 -- 172.21.15.26:0/4215669972 learned_addr learned my addr 172.21.15.26:0/4215669972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 -- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8460145f40 msgr2=0x7f846009d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8460145f40 0x7f846009d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 -- 172.21.15.26:0/4215669972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84600af980 msgr2=0x7f8460159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84600af980 0x7f8460159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 -- 172.21.15.26:0/4215669972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84500094a0 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.459+0000 7f84652fc700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84500090b0 tx=0x7f8450004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.460+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450004590 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.460+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845001f070 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.460+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450003af0 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.460+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846009ccc0 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.460+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846009d280 con 0x7f8460159870 2024-01-16T14:54:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.461+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8460075480 con 0x7f8460159870 2024-01-16T14:54:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.463+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8450003c50 con 0x7f8460159870 2024-01-16T14:54:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.463+0000 7f84567fc700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 0x7f844406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.463+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8450021070 con 0x7f8460159870 2024-01-16T14:54:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.464+0000 7f8465afd700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 0x7f844406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.465+0000 7fea6e0ae700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 0x7fea4c06d8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea68069500 tx=0x7fea6401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.464+0000 7f8465afd700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 0x7f844406d840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8460071590 tx=0x7f845c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.469+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84500563c0 con 0x7f8460159870 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.462+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/2953090907 >> v2:172.21.15.26:3300/0 conn(0x7f27f80722c0 0x7f27f80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.462+0000 7f27f6ffd700 1 -- 172.21.15.26:0/2953090907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f81440f0 msgr2=0x7f27f806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.462+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/2953090907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f81440f0 0x7f27f806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.462+0000 7f27f6ffd700 1 -- 172.21.15.26:0/2953090907 >> v1:172.21.15.93:6789/0 conn(0x7f27f8143ae0 legacy=0x7f27f8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.462+0000 7f27f6ffd700 1 -- 172.21.15.26:0/2953090907 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80b3dc0 con 0x7f27f80722c0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.463+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/2953090907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f80722c0 0x7f27f80b51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009840 tx=0x7f27e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d45e6a49b79374 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.464+0000 7f27f67fc700 1 -- 172.21.15.26:0/2953090907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8011830 con 0x7f27f80722c0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.464+0000 7f27f67fc700 1 -- 172.21.15.26:0/2953090907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8011990 con 0x7f27f80722c0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.464+0000 7f27f67fc700 1 -- 172.21.15.26:0/2953090907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8011d20 con 0x7f27f80722c0 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.465+0000 7f27fdc2f700 1 -- 172.21.15.26:0/2953090907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f80722c0 msgr2=0x7f27f80b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.465+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/2953090907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f80722c0 0x7f27f80b51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009840 tx=0x7f27e8009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.473+0000 7f27fdc2f700 1 -- 172.21.15.26:0/2953090907 shutdown_connections 2024-01-16T14:54:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.473+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/2953090907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f81440f0 0x7f27f806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.473+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/2953090907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f80722c0 0x7f27f80b51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.473+0000 7f27fdc2f700 1 -- 172.21.15.26:0/2953090907 >> 172.21.15.26:0/2953090907 conn(0x7f27f80b2740 msgr2=0x7f27f80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.473+0000 7f27fdc2f700 1 -- 172.21.15.26:0/2953090907 shutdown_connections 2024-01-16T14:54:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.484+0000 7f27fdc2f700 1 -- 172.21.15.26:0/2953090907 wait complete. 2024-01-16T14:54:52.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.486+0000 7f27fdc2f700 1 Processor -- start 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.486+0000 7f27fdc2f700 1 -- start start 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f80722c0 0x7f27f813ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 0x7f27f815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 0x7f27f815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8204160 con 0x7f27f81440f0 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8204440 con 0x7f27f8143ae0 2024-01-16T14:54:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.488+0000 7f27fdc2f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8204720 con 0x7f27f80722c0 2024-01-16T14:54:52.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f77fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f80722c0 0x7f27f813ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 0x7f27f815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 0x7f27f815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 0x7f27f815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49740/0 (socket says 172.21.15.26:49740) 2024-01-16T14:54:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f7fff700 1 -- 172.21.15.26:0/4125054253 learned_addr learned my addr 172.21.15.26:0/4125054253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 -- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f80722c0 msgr2=0x7f27f813ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f80722c0 0x7f27f813ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 -- 172.21.15.26:0/4125054253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 msgr2=0x7f27f815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 0x7f27f815d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.498+0000 7f27f6ffd700 1 -- 172.21.15.26:0/4125054253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80094a0 con 0x7f27f8143ae0 2024-01-16T14:54:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.501+0000 7f27f6ffd700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 0x7f27f815d3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27e8016000 tx=0x7f27e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.502+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e80036a0 con 0x7f27f8143ae0 2024-01-16T14:54:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.502+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e801f070 con 0x7f27f8143ae0 2024-01-16T14:54:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.502+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003a00 con 0x7f27f8143ae0 2024-01-16T14:54:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.502+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f8204a00 con 0x7f27f8143ae0 2024-01-16T14:54:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.502+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f813f130 con 0x7f27f8143ae0 2024-01-16T14:54:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.508+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f27e8003b60 con 0x7f27f8143ae0 2024-01-16T14:54:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.508+0000 7f27dffff700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 0x7f27e406d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.508+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27e8021070 con 0x7f27f8143ae0 2024-01-16T14:54:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.510+0000 7f27f77fe700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 0x7f27e406d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.511+0000 7f27f77fe700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 0x7f27e406d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005910 tx=0x7f27ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.515+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d80052f0 con 0x7f27f8143ae0 2024-01-16T14:54:52.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.521+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27e80562c0 con 0x7f27f8143ae0 2024-01-16T14:54:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.868+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fed381603a0 con 0x7fed38165f70 2024-01-16T14:54:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.869+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f32700055a0 con 0x7f3290145870 2024-01-16T14:54:52.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.870+0000 7fed357fa700 1 -- 172.21.15.26:0/4255867490 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed381603a0 con 0x7fed38165f70 2024-01-16T14:54:52.853 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-01-16T14:54:52.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.870+0000 7f327effd700 1 -- 172.21.15.26:0/2476391434 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3280017090 con 0x7f3290145870 2024-01-16T14:54:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4870005870 con 0x7f489015e3b0 2024-01-16T14:54:52.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2814142390 con 0x7f28141446a0 2024-01-16T14:54:52.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 msgr2=0x7f327806dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 0x7f327806dc70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3288014040 tx=0x7f3288011000 comp rx=0 tx=0).stop 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 msgr2=0x7f329015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 0x7f329015de30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f328000c490 tx=0x7f328000c7a0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 msgr2=0x7fed2406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 0x7fed2406d810 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed28006b20 tx=0x7fed28006ab0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 msgr2=0x7fed38160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fed3000b9e0 tx=0x7fed3000bda0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 shutdown_connections 2024-01-16T14:54:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fed38165f70 0x7fed38160d40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fed2406b350 0x7fed2406d810 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed380cbb80 0x7fed38164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 --2- 172.21.15.26:0/4255867490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fed3814bf10 0x7fed3815f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.874+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 >> 172.21.15.26:0/4255867490 conn(0x7fed380cf980 msgr2=0x7fed3814b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.875+0000 7f488d7fa700 1 -- 172.21.15.26:0/542513902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48800586b0 con 0x7f489015e3b0 2024-01-16T14:54:52.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.875+0000 7f2810ff9700 1 -- 172.21.15.26:0/3053386807 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2814142390 con 0x7f28141446a0 2024-01-16T14:54:52.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.875+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 shutdown_connections 2024-01-16T14:54:52.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.875+0000 7fed3f22c700 1 -- 172.21.15.26:0/4255867490 wait complete. 2024-01-16T14:54:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 shutdown_connections 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f327806b7b0 0x7f327806dc70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3290166600 0x7f329015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290070a80 0x7f3290144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 --2- 172.21.15.26:0/2476391434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3290145870 0x7f329015de30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 >> 172.21.15.26:0/2476391434 conn(0x7f32900cf980 msgr2=0x7f32900cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 shutdown_connections 2024-01-16T14:54:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.878+0000 7f32948ef700 1 -- 172.21.15.26:0/2476391434 wait complete. 2024-01-16T14:54:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.880+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 msgr2=0x7f280006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.881+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 0x7f280006d8e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2804000f60 tx=0x7f2804007eb0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.881+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 msgr2=0x7f28141435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.881+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f280c014040 tx=0x7f280c004ad0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 shutdown_connections 2024-01-16T14:54:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2814143ae0 0x7f281415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f280006b420 0x7f280006d8e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28140cb1d0 0x7f28141413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 --2- 172.21.15.26:0/3053386807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28141446a0 0x7f28141435a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 >> 172.21.15.26:0/3053386807 conn(0x7f28140cf980 msgr2=0x7f28140b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 msgr2=0x7f487c06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 0x7f487c06d8e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f488400a810 tx=0x7f4884008040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 msgr2=0x7f48900b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f488000b3e0 tx=0x7f488000b7a0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 shutdown_connections 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f487c06b420 0x7f487c06d8e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f489015e3b0 0x7f48900b0fb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4890071bb0 0x7f489014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 --2- 172.21.15.26:0/542513902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48900d1650 0x7f48901f8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.882+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 >> 172.21.15.26:0/542513902 conn(0x7f48900d0b30 msgr2=0x7f489015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.883+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 shutdown_connections 2024-01-16T14:54:52.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.883+0000 7f489709b700 1 -- 172.21.15.26:0/542513902 wait complete. 2024-01-16T14:54:52.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.886+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6e100cf500 con 0x7f6e1015b6a0 2024-01-16T14:54:52.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.887+0000 7f6df77fe700 1 -- 172.21.15.26:0/1582765878 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e0401d090 con 0x7f6e1015b6a0 2024-01-16T14:54:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:55834574852 2024-01-16T14:54:52.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.888+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 shutdown_connections 2024-01-16T14:54:52.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.888+0000 7f281a872700 1 -- 172.21.15.26:0/3053386807 wait complete. 2024-01-16T14:54:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.890+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 msgr2=0x7f6dfc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.890+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 0x7f6dfc06d840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000c1e0 tx=0x7f6e00011000 comp rx=0 tx=0).stop 2024-01-16T14:54:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.890+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 msgr2=0x7f6e100cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.890+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 0x7f6e100cafa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e1006a6a0 tx=0x7f6e0400dbf0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 shutdown_connections 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dfc06b380 0x7f6dfc06d840 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e10200810 0x7f6e100cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e1015b2c0 0x7f6e100b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 --2- 172.21.15.26:0/1582765878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e1015b6a0 0x7f6e100cafa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.892+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 >> 172.21.15.26:0/1582765878 conn(0x7f6e100cf980 msgr2=0x7f6e100af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.893+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 shutdown_connections 2024-01-16T14:54:52.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.893+0000 7f6e1566d700 1 -- 172.21.15.26:0/1582765878 wait complete. 2024-01-16T14:54:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:52.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-01-16T14:54:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.894+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fea50005870 con 0x7fea680ce440 2024-01-16T14:54:52.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.895+0000 7fea5effd700 1 -- 172.21.15.26:0/518537493 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea580092a0 con 0x7fea680ce440 2024-01-16T14:54:52.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-01-16T14:54:52.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 msgr2=0x7fea4c06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 0x7fea4c06d8e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea68069500 tx=0x7fea6401c040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 msgr2=0x7fea681576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fea5800b4a0 tx=0x7fea5800b7b0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 shutdown_connections 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea680b2740 0x7fea680b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea68071190 0x7fea680b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fea4c06b420 0x7fea4c06d8e0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea68069500 tx=0x7fea6401c040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 --2- 172.21.15.26:0/518537493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea680ce440 0x7fea681576b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.904+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 >> 172.21.15.26:0/518537493 conn(0x7fea680cf980 msgr2=0x7fea680b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.907+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 shutdown_connections 2024-01-16T14:54:52.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.907+0000 7fea70312700 1 -- 172.21.15.26:0/518537493 wait complete. 2024-01-16T14:54:52.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.910+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f846015b420 con 0x7f8460159870 2024-01-16T14:54:52.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.912+0000 7f84567fc700 1 -- 172.21.15.26:0/4215669972 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f845002b020 con 0x7f8460159870 2024-01-16T14:54:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:52.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 msgr2=0x7f844406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 0x7f844406d840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8460071590 tx=0x7f845c008040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 msgr2=0x7f84601464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84500090b0 tx=0x7f8450004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 shutdown_connections 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8460145f40 0x7f846009d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f844406b380 0x7f844406d840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84600af980 0x7f8460159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 --2- 172.21.15.26:0/4215669972 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8460159870 0x7f84601464d0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f84500090b0 tx=0x7f8450004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.916+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 >> 172.21.15.26:0/4215669972 conn(0x7f84600b5070 msgr2=0x7f8460159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-01-16T14:54:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.917+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 shutdown_connections 2024-01-16T14:54:52.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.917+0000 7f8467d61700 1 -- 172.21.15.26:0/4215669972 wait complete. 2024-01-16T14:54:52.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-01-16T14:54:52.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-01-16T14:54:52.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-01-16T14:54:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.949+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f27d80055a0 con 0x7f27f8143ae0 2024-01-16T14:54:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.950+0000 7f27dffff700 1 -- 172.21.15.26:0/4125054253 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f27e8015020 con 0x7f27f8143ae0 2024-01-16T14:54:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 msgr2=0x7f27e406d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 0x7f27e406d8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005910 tx=0x7f27ec009040 comp rx=0 tx=0).stop 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 msgr2=0x7f27f815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 0x7f27f815d3d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27e8016000 tx=0x7f27e8004750 comp rx=0 tx=0).stop 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 shutdown_connections 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f27e406b420 0x7f27e406d8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27f81440f0 0x7f27f815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27f80722c0 0x7f27f813ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 --2- 172.21.15.26:0/4125054253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27f8143ae0 0x7f27f815d3d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.952+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 >> 172.21.15.26:0/4125054253 conn(0x7f27f80b2740 msgr2=0x7f27f80ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.953+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 shutdown_connections 2024-01-16T14:54:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:52.953+0000 7f27fdc2f700 1 -- 172.21.15.26:0/4125054253 wait complete. 2024-01-16T14:54:52.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-01-16T14:54:53.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-16T14:54:53.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-16T14:54:53.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-16T14:54:53.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-16T14:54:53.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-16T14:54:53.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-16T14:54:53.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-16T14:54:53.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-16T14:54:54.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.115+0000 7f6b89f34700 1 Processor -- start 2024-01-16T14:54:54.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.115+0000 7f6b89f34700 1 -- start start 2024-01-16T14:54:54.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.117+0000 7f6b89f34700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b84070a80 0x7f6b84070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.117+0000 7f6b89f34700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 0x7f6b84165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.117+0000 7f6b89f34700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b840ca8e0 con 0x7f6b84166430 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.117+0000 7f6b89f34700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840713a0 con 0x7f6b84070a80 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.117+0000 7f6b89f34700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840c9fd0 con 0x7f6b84145870 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b82ffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 0x7f6b84165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b82ffd700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 0x7f6b84165ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:47786/0 (socket says 172.21.15.26:47786) 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b82ffd700 1 -- 172.21.15.26:0/2618955117 learned_addr learned my addr 172.21.15.26:0/2618955117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b837fe700 1 --2- 172.21.15.26:0/2618955117 >> v2:172.21.15.26:3300/0 conn(0x7f6b84070a80 0x7f6b84070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b837fe700 1 -- 172.21.15.26:0/2618955117 >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 msgr2=0x7f6b84165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b837fe700 1 --2- 172.21.15.26:0/2618955117 >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 0x7f6b84165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b837fe700 1 -- 172.21.15.26:0/2618955117 >> v1:172.21.15.93:6789/0 conn(0x7f6b84166430 legacy=0x7f6b840cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.118+0000 7f6b837fe700 1 -- 172.21.15.26:0/2618955117 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84166c60 con 0x7f6b84070a80 2024-01-16T14:54:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.119+0000 7f6b837fe700 1 --2- 172.21.15.26:0/2618955117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84070a80 0x7f6b84070e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b84070650 tx=0x7f6b78009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895e09f2d448500b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.120+0000 7f6b827fc700 1 -- 172.21.15.26:0/2618955117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78004230 con 0x7f6b84070a80 2024-01-16T14:54:54.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.120+0000 7f6b827fc700 1 -- 172.21.15.26:0/2618955117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b78004390 con 0x7f6b84070a80 2024-01-16T14:54:54.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.120+0000 7f6b827fc700 1 -- 172.21.15.26:0/2618955117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780036a0 con 0x7f6b84070a80 2024-01-16T14:54:54.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.121+0000 7f6b89f34700 1 -- 172.21.15.26:0/2618955117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84070a80 msgr2=0x7f6b84070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.121+0000 7f6b89f34700 1 --2- 172.21.15.26:0/2618955117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84070a80 0x7f6b84070e60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b84070650 tx=0x7f6b78009b20 comp rx=0 tx=0).stop 2024-01-16T14:54:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 -- 172.21.15.26:0/2618955117 shutdown_connections 2024-01-16T14:54:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 --2- 172.21.15.26:0/2618955117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84070a80 0x7f6b84070e60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 --2- 172.21.15.26:0/2618955117 >> v2:172.21.15.93:3300/0 conn(0x7f6b84145870 0x7f6b84165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 -- 172.21.15.26:0/2618955117 >> 172.21.15.26:0/2618955117 conn(0x7f6b840cf980 msgr2=0x7f6b840cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 -- 172.21.15.26:0/2618955117 shutdown_connections 2024-01-16T14:54:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.122+0000 7f6b89f34700 1 -- 172.21.15.26:0/2618955117 wait complete. 2024-01-16T14:54:54.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.124+0000 7f6b89f34700 1 Processor -- start 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.125+0000 7f6b89f34700 1 -- start start 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b84070a80 0x7f6b84144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b84166430 0x7f6b84132ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84137080 con 0x7f6b84145870 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b841367e0 con 0x7f6b84166430 2024-01-16T14:54:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b89f34700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b84136ac0 con 0x7f6b84070a80 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b82ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b82ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49754/0 (socket says 172.21.15.26:49754) 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.127+0000 7f6b82ffd700 1 -- 172.21.15.26:0/986737536 learned_addr learned my addr 172.21.15.26:0/986737536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 -- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b84070a80 msgr2=0x7f6b84144cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b84070a80 0x7f6b84144cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 -- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b84166430 msgr2=0x7f6b84132ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b84166430 0x7f6b84132ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 -- 172.21.15.26:0/986737536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780094a0 con 0x7f6b84145870 2024-01-16T14:54:54.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.128+0000 7f6b82ffd700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400e7e0 tx=0x7f6b7400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.130+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74013610 con 0x7f6b84145870 2024-01-16T14:54:54.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.130+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b74013770 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.130+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74013b20 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.130+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b84136da0 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.130+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8415bd80 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.135+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6b74013c80 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.135+0000 7f6b6bfff700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 0x7f6b7006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.135+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b74089850 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.136+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b640052f0 con 0x7f6b84145870 2024-01-16T14:54:54.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.136+0000 7f6b837fe700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 0x7f6b7006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.147+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b740546a0 con 0x7f6b84145870 2024-01-16T14:54:54.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.147+0000 7f6b837fe700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 0x7f6b7006d8e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b840b5970 tx=0x7f6b78010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.180+0000 7f0bfa62c700 1 Processor -- start 2024-01-16T14:54:54.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.180+0000 7f0bfa62c700 1 -- start start 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.181+0000 7f0bfa62c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bf4070a80 0x7f0bf4070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.181+0000 7f0bfa62c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4166600 0x7f0bf40cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.181+0000 7f0bfa62c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf4166a30 con 0x7f0bf4145870 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.181+0000 7f0bfa62c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40ca8e0 con 0x7f0bf4070a80 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.181+0000 7f0bfa62c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40713a0 con 0x7f0bf4166600 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bf4070a80 0x7f0bf4070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bf4070a80 0x7f0bf4070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49764/0 (socket says 172.21.15.26:49764) 2024-01-16T14:54:54.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 -- 172.21.15.26:0/3581267403 learned_addr learned my addr 172.21.15.26:0/3581267403 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 -- 172.21.15.26:0/3581267403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4166600 msgr2=0x7f0bf40cad70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 --2- 172.21.15.26:0/3581267403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4166600 0x7f0bf40cad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 -- 172.21.15.26:0/3581267403 >> v1:172.21.15.26:6789/0 conn(0x7f0bf4145870 legacy=0x7f0bf4165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.182+0000 7f0bf3fff700 1 -- 172.21.15.26:0/3581267403 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40c9fd0 con 0x7f0bf4070a80 2024-01-16T14:54:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.183+0000 7f0bf3fff700 1 --2- 172.21.15.26:0/3581267403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4070a80 0x7f0bf4070e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0be40098f0 tx=0x7f0be4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbbe335fd971f6b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.184+0000 7f1f0f5e8700 1 Processor -- start 2024-01-16T14:54:54.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.184+0000 7f1f0f5e8700 1 -- start start 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.185+0000 7f0bf2ffd700 1 -- 172.21.15.26:0/3581267403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4004230 con 0x7f0bf4070a80 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.185+0000 7f0bf2ffd700 1 -- 172.21.15.26:0/3581267403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4004390 con 0x7f0bf4070a80 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.185+0000 7f0bf2ffd700 1 -- 172.21.15.26:0/3581267403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40036a0 con 0x7f0bf4070a80 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0f5e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3581267403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4070a80 msgr2=0x7f0bf4070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3581267403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4070a80 0x7f0bf4070e60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0be40098f0 tx=0x7f0be4009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0f5e8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080ce000 con 0x7f1f0815b170 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0f5e8700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080ce2c0 con 0x7f1f080d04a0 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0f5e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080ce5a0 con 0x7f1f08071390 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0d384700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0d384700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38568/0 (socket says 172.21.15.26:38568) 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.186+0000 7f1f0d384700 1 -- 172.21.15.26:0/3022146087 learned_addr learned my addr 172.21.15.26:0/3022146087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f1f0d384700 1 -- 172.21.15.26:0/3022146087 >> v1:172.21.15.93:6789/0 conn(0x7f1f080d04a0 legacy=0x7f1f0815aa60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f1f0d384700 1 -- 172.21.15.26:0/3022146087 >> v1:172.21.15.26:6789/0 conn(0x7f1f0815b170 legacy=0x7f1f080ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f1f0d384700 1 -- 172.21.15.26:0/3022146087 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080ce880 con 0x7f1f08071390 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3581267403 shutdown_connections 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3581267403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4166600 0x7f0bf40cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3581267403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4070a80 0x7f0bf4070e60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3581267403 >> 172.21.15.26:0/3581267403 conn(0x7f0bf40cf980 msgr2=0x7f0bf40cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7fb599a12700 1 Processor -- start 2024-01-16T14:54:54.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f1f0d384700 1 --2- 172.21.15.26:0/3022146087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009e80 tx=0x7f1f0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd07569b65747121 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3581267403 shutdown_connections 2024-01-16T14:54:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.187+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3581267403 wait complete. 2024-01-16T14:54:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.189+0000 7fb599a12700 1 -- start start 2024-01-16T14:54:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.188+0000 7f1efffff700 1 -- 172.21.15.26:0/3022146087 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000e070 con 0x7f1f08071390 2024-01-16T14:54:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.189+0000 7f1efffff700 1 -- 172.21.15.26:0/3022146087 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00003a20 con 0x7f1f08071390 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.189+0000 7f1efffff700 1 -- 172.21.15.26:0/3022146087 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00003dd0 con 0x7f1f08071390 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.189+0000 7f0bfa62c700 1 Processor -- start 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb599a12700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415db10 0x7fb59415def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb599a12700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb594071390 0x7fb59415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb599a12700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5940ce000 con 0x7fb59415a520 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb599a12700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940cb2d0 con 0x7fb594071390 2024-01-16T14:54:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb599a12700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940ce2c0 con 0x7fb59415db10 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/3022146087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 msgr2=0x7f1f0815a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/3022146087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009e80 tx=0x7f1f0000b330 comp rx=0 tx=0).stop 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/3022146087 shutdown_connections 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/3022146087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f08071390 0x7f1f0815a520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/3022146087 >> 172.21.15.26:0/3022146087 conn(0x7f1f080cf980 msgr2=0x7f1f080cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7f0bfa62c700 1 -- start start 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb5937fe700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fb59415a520 0x7fb59415a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:49772/0 (socket says 172.21.15.26:49772) 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb5937fe700 1 -- 172.21.15.26:0/2280357058 learned_addr learned my addr 172.21.15.26:0/2280357058 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.190+0000 7fb592ffd700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415db10 0x7fb59415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7fb5927fc700 1 --2- 172.21.15.26:0/2280357058 >> v2:172.21.15.26:3300/0 conn(0x7fb594071390 0x7fb59415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4070a80 0x7f0bf4137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 0x7f0bf4165250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4137f80 con 0x7f0bf4166600 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4165790 con 0x7f0bf4145870 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bfa62c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf41ed1c0 con 0x7f0bf4070a80 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7fb5927fc700 1 -- 172.21.15.26:0/2280357058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415db10 msgr2=0x7fb59415def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7fb5927fc700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415db10 0x7fb59415def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7fb5927fc700 1 -- 172.21.15.26:0/2280357058 >> v1:172.21.15.93:6789/0 conn(0x7fb59415a520 legacy=0x7fb59415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7fb5927fc700 1 -- 172.21.15.26:0/2280357058 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940ce5a0 con 0x7fb594071390 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f0bf3fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4070a80 0x7f0bf4137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f0bf8bc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f0bf8bc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49784/0 (socket says 172.21.15.26:49784) 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f0bf8bc9700 1 -- 172.21.15.26:0/3481388082 learned_addr learned my addr 172.21.15.26:0/3481388082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7fb5927fc700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb594071390 0x7fb59415e430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb58400b1b0 tx=0x7fb58400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7bc914b1b1b745 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f115fdc4700 1 Processor -- start 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.191+0000 7f115fdc4700 1 -- start start 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115fdc4700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f11580c9fd0 0x7f115815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115fdc4700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f115806cae0 0x7f115806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115fdc4700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580cded0 con 0x7f11580cef70 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115fdc4700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580ce1b0 con 0x7f115806cae0 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115fdc4700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11580ce490 con 0x7f11580c9fd0 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7fb591ffb700 1 -- 172.21.15.26:0/2280357058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584004030 con 0x7fb594071390 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7fb591ffb700 1 -- 172.21.15.26:0/2280357058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584004190 con 0x7fb594071390 2024-01-16T14:54:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7fb591ffb700 1 -- 172.21.15.26:0/2280357058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584004520 con 0x7fb594071390 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115e361700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f115806cae0 0x7f115806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115e361700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f115806cae0 0x7f115806cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49786/0 (socket says 172.21.15.26:49786) 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115e361700 1 -- 172.21.15.26:0/367158186 learned_addr learned my addr 172.21.15.26:0/367158186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.192+0000 7f115db60700 1 --2- 172.21.15.26:0/367158186 >> v2:172.21.15.93:3300/0 conn(0x7f11580c9fd0 0x7f115815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115cb5e700 1 -- 172.21.15.26:0/367158186 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552031532 0 0) 0x7f11580cded0 con 0x7f11580cef70 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115cb5e700 1 -- 172.21.15.26:0/367158186 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11440035b0 con 0x7f11580cef70 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115e361700 1 -- 172.21.15.26:0/367158186 >> v2:172.21.15.93:3300/0 conn(0x7f11580c9fd0 msgr2=0x7f115815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115e361700 1 --2- 172.21.15.26:0/367158186 >> v2:172.21.15.93:3300/0 conn(0x7f11580c9fd0 0x7f115815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115e361700 1 -- 172.21.15.26:0/367158186 >> v1:172.21.15.93:6789/0 conn(0x7f11580cef70 legacy=0x7f11580ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115e361700 1 -- 172.21.15.26:0/367158186 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580ce770 con 0x7f115806cae0 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/3022146087 shutdown_connections 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 -- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4070a80 msgr2=0x7f0bf4137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf37fe700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 0x7f0bf4165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4070a80 0x7f0bf4137a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 -- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 msgr2=0x7f0bf4165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 0x7f0bf4165250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 -- 172.21.15.26:0/3481388082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009550 con 0x7f0bf4166600 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf37fe700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 0x7f0bf4165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 -- 172.21.15.26:0/2280357058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb594071390 msgr2=0x7fb59415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb594071390 0x7fb59415e430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb58400b1b0 tx=0x7fb58400b4c0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 -- 172.21.15.26:0/2280357058 shutdown_connections 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415db10 0x7fb59415def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 --2- 172.21.15.26:0/2280357058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb594071390 0x7fb59415e430 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb58400b1b0 tx=0x7fb58400b4c0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7fb599a12700 1 -- 172.21.15.26:0/2280357058 >> 172.21.15.26:0/2280357058 conn(0x7fb5940cf980 msgr2=0x7fb5940cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f115e361700 1 --2- 172.21.15.26:0/367158186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f115806cae0 0x7f115806cf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1158069290 tx=0x7f115000d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844730762a62b756 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f0bf8bc9700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00b990 tx=0x7f0bec00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7fb599a12700 1 -- 172.21.15.26:0/2280357058 shutdown_connections 2024-01-16T14:54:54.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7fb599a12700 1 -- 172.21.15.26:0/2280357058 wait complete. 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00c930 con 0x7f0bf4166600 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00ca90 con 0x7f0bf4166600 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00ce40 con 0x7f0bf4166600 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.193+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/3022146087 wait complete. 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.195+0000 7f1f0f5e8700 1 Processor -- start 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.195+0000 7f115cb5e700 1 -- 172.21.15.26:0/367158186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150010070 con 0x7f115806cae0 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.195+0000 7f115cb5e700 1 -- 172.21.15.26:0/367158186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000dc30 con 0x7f115806cae0 2024-01-16T14:54:54.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.195+0000 7f115cb5e700 1 -- 172.21.15.26:0/367158186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150007b30 con 0x7f115806cae0 2024-01-16T14:54:54.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf41ed770 con 0x7f0bf4166600 2024-01-16T14:54:54.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.194+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf41edd30 con 0x7f0bf4166600 2024-01-16T14:54:54.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7f115fdc4700 1 -- 172.21.15.26:0/367158186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f115806cae0 msgr2=0x7f115806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7f115fdc4700 1 --2- 172.21.15.26:0/367158186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f115806cae0 0x7f115806cf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1158069290 tx=0x7f115000d650 comp rx=0 tx=0).stop 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7fb599a12700 1 Processor -- start 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7fb599a12700 1 -- start start 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf41edf50 con 0x7f0bf4166600 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.196+0000 7f1f0f5e8700 1 -- start start 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 0x7f1f08159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f080d04a0 0x7f1f080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 0x7f1f080c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c86a0 con 0x7f1f08071390 2024-01-16T14:54:54.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c8980 con 0x7f1f080d04a0 2024-01-16T14:54:54.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0f5e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080c8c60 con 0x7f1f0815b170 2024-01-16T14:54:54.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415db10 0x7fb59413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb59413e690 0x7fb59415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940b2ef0 con 0x7fb59415db10 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59413eff0 con 0x7fb59413e690 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7fb599a12700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59413f2d0 con 0x7fb59415a520 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38620/0 (socket says 172.21.15.26:38620) 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 -- 172.21.15.26:0/2615956198 learned_addr learned my addr 172.21.15.26:0/2615956198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.197+0000 7f1f0db85700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 0x7f1f080c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 0x7f1f08159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0db85700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 0x7f1f080c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38606/0 (socket says 172.21.15.26:38606) 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0db85700 1 -- 172.21.15.26:0/974987830 learned_addr learned my addr 172.21.15.26:0/974987830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0cb83700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f080d04a0 0x7f1f080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 -- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb59413e690 msgr2=0x7fb59415bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 -- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 msgr2=0x7f1f080c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 0x7f1f080c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 -- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f080d04a0 msgr2=0x7f1f080c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f080d04a0 0x7f1f080c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb59413e690 0x7fb59415bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 -- 172.21.15.26:0/2615956198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415db10 msgr2=0x7fb59413e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415db10 0x7fb59413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7fb592ffd700 1 -- 172.21.15.26:0/2615956198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584009d30 con 0x7fb59415a520 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.198+0000 7f1f0d384700 1 -- 172.21.15.26:0/974987830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00009b30 con 0x7f1f08071390 2024-01-16T14:54:54.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f158cc06700 1 Processor -- start 2024-01-16T14:54:54.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb592ffd700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb58800d800 tx=0x7fb58800dbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f115fdc4700 1 -- 172.21.15.26:0/367158186 shutdown_connections 2024-01-16T14:54:54.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f115fdc4700 1 --2- 172.21.15.26:0/367158186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f115806cae0 0x7f115806cf70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f115fdc4700 1 --2- 172.21.15.26:0/367158186 >> v2:172.21.15.93:3300/0 conn(0x7f11580c9fd0 0x7f115815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f115fdc4700 1 -- 172.21.15.26:0/367158186 >> 172.21.15.26:0/367158186 conn(0x7f11580cf980 msgr2=0x7f11580cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f1f0d384700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 0x7f1f08159640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f00014000 tx=0x7f1f00004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880125f0 con 0x7fb59415a520 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588012750 con 0x7fb59415a520 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588012ae0 con 0x7fb59415a520 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59413f5b0 con 0x7fb59415a520 2024-01-16T14:54:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59413c8a0 con 0x7fb59415a520 2024-01-16T14:54:54.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0bec006d50 con 0x7f0bf4166600 2024-01-16T14:54:54.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f0bf0ff9700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 0x7f0be006d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bec089a70 con 0x7f0bf4166600 2024-01-16T14:54:54.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740052f0 con 0x7fb59415a520 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.199+0000 7f158cc06700 1 -- start start 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.200+0000 7f158cc06700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.200+0000 7f158cc06700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f158806da60 0x7f158815c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.200+0000 7f158cc06700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158815b090 con 0x7f15880af720 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.200+0000 7f158cc06700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158806bf20 con 0x7f158806da60 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.200+0000 7f158cc06700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158815b370 con 0x7f15880afb00 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38626/0 (socket says 172.21.15.26:38626) 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 -- 172.21.15.26:0/902219955 learned_addr learned my addr 172.21.15.26:0/902219955 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 -- 172.21.15.26:0/902219955 >> v2:172.21.15.93:3300/0 conn(0x7f158806da60 msgr2=0x7f158815c150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 --2- 172.21.15.26:0/902219955 >> v2:172.21.15.93:3300/0 conn(0x7f158806da60 0x7f158815c150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 -- 172.21.15.26:0/902219955 >> v1:172.21.15.26:6789/0 conn(0x7f15880af720 legacy=0x7f158806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.201+0000 7f1585d9b700 1 -- 172.21.15.26:0/902219955 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158815b650 con 0x7f15880afb00 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f1585d9b700 1 --2- 172.21.15.26:0/902219955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f157c010cb0 tx=0x7f157c00b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64f238a3213f453 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f115fdc4700 1 -- 172.21.15.26:0/367158186 shutdown_connections 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb588012c40 con 0x7fb59415a520 2024-01-16T14:54:54.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f0bf3fff700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 0x7f0be006d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f158559a700 1 -- 172.21.15.26:0/902219955 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00be60 con 0x7f15880afb00 2024-01-16T14:54:54.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f158559a700 1 -- 172.21.15.26:0/902219955 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c018410 con 0x7f15880afb00 2024-01-16T14:54:54.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7f158559a700 1 -- 172.21.15.26:0/902219955 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c0187c0 con 0x7f15880afb00 2024-01-16T14:54:54.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7fb57bfff700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 0x7fb57c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.202+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb588089f90 con 0x7fb59415a520 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 -- 172.21.15.26:0/902219955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 msgr2=0x7f158806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 --2- 172.21.15.26:0/902219955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f157c010cb0 tx=0x7f157c00b4e0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 -- 172.21.15.26:0/902219955 shutdown_connections 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 --2- 172.21.15.26:0/902219955 >> v2:172.21.15.93:3300/0 conn(0x7f158806da60 0x7f158815c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 --2- 172.21.15.26:0/902219955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15880afb00 0x7f158806b9e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f158cc06700 1 -- 172.21.15.26:0/902219955 >> 172.21.15.26:0/902219955 conn(0x7f15880717f0 msgr2=0x7f15880b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.204+0000 7f0bf3fff700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 0x7f0be006d8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009c40 tx=0x7f0be4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.204+0000 7f158cc06700 1 -- 172.21.15.26:0/902219955 shutdown_connections 2024-01-16T14:54:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.203+0000 7f115fdc4700 1 -- 172.21.15.26:0/367158186 wait complete. 2024-01-16T14:54:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.205+0000 7fb5927fc700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 0x7fb57c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.205+0000 7f115fdc4700 1 Processor -- start 2024-01-16T14:54:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.205+0000 7fb5927fc700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 0x7fb57c06da80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb594069500 tx=0x7fb584010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.204+0000 7f158cc06700 1 -- 172.21.15.26:0/902219955 wait complete. 2024-01-16T14:54:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.206+0000 7f158cc06700 1 Processor -- start 2024-01-16T14:54:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.206+0000 7f115fdc4700 1 -- start start 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 0x7f11580b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 0x7f11580b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158141590 con 0x7f11580cef70 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158154830 con 0x7f11580c9fd0 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115fdc4700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b3b10 con 0x7f115806cae0 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115e361700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb588054de0 con 0x7fb59415a520 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115e361700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49816/0 (socket says 172.21.15.26:49816) 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115e361700 1 -- 172.21.15.26:0/4070430716 learned_addr learned my addr 172.21.15.26:0/4070430716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115d35f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 0x7f11580b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f115db60700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 0x7f11580b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.206+0000 7f158cc06700 1 -- start start 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f158806da60 0x7f15881fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 0x7f15881e5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 0x7f15881543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880cddd0 con 0x7f15880af720 2024-01-16T14:54:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881fc200 con 0x7f15880ccc00 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f158cc06700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158809cc90 con 0x7f158806da60 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f1585d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 0x7f15881e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f1585d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 0x7f15881e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49810/0 (socket says 172.21.15.26:49810) 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f1585d9b700 1 -- 172.21.15.26:0/70776903 learned_addr learned my addr 172.21.15.26:0/70776903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.208+0000 7f771ff03700 1 Processor -- start 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f771ff03700 1 -- start start 2024-01-16T14:54:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 0x7f15881543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 -- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f158806da60 msgr2=0x7f15881fbcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f158806da60 0x7f15881fbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 -- 172.21.15.26:0/70776903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 msgr2=0x7f15881e5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 0x7f15881e5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 -- 172.21.15.26:0/70776903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c010910 con 0x7f15880ccc00 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 -- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 msgr2=0x7f11580b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 0x7f11580b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 -- 172.21.15.26:0/4070430716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 msgr2=0x7f1158142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 -- 172.21.15.26:0/4070430716 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115000d050 con 0x7f11580c9fd0 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115db60700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 0x7f11580b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115e361700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115d35f700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 0x7f11580b34e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f115400c450 tx=0x7f115400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540052d0 con 0x7f11580c9fd0 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580b3df0 con 0x7f11580c9fd0 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1158141220 con 0x7f11580c9fd0 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004440 con 0x7f1f08071390 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f000092e0 con 0x7f1f08071390 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00004800 con 0x7f1f08071390 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154005430 con 0x7f11580c9fd0 2024-01-16T14:54:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540038c0 con 0x7f11580c9fd0 2024-01-16T14:54:54.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771ff03700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7718162770 0x7f771806bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771ff03700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 0x7f7718162b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771ff03700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771806b9a0 con 0x7f771806c4a0 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771ff03700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771806b070 con 0x7f7718162770 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771ff03700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771806b350 con 0x7f77180b19b0 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771e4a0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 0x7f7718162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771dc9f700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7718162770 0x7f771806bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771e4a0700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 0x7f7718162b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:38652/0 (socket says 172.21.15.26:38652) 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f771e4a0700 1 -- 172.21.15.26:0/1910810362 learned_addr learned my addr 172.21.15.26:0/1910810362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.211+0000 7f771dc9f700 1 -- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 msgr2=0x7f7718162b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.211+0000 7f771dc9f700 1 --2- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 0x7f7718162b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.211+0000 7f771dc9f700 1 -- 172.21.15.26:0/1910810362 >> v1:172.21.15.93:6789/0 conn(0x7f771806c4a0 legacy=0x7f77180b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.211+0000 7f771dc9f700 1 -- 172.21.15.26:0/1910810362 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771806b630 con 0x7f7718162770 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.211+0000 7f771dc9f700 1 --2- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f771806bf60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f770c009840 tx=0x7f770c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84b917e514e423d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771cc9d700 1 -- 172.21.15.26:0/1910810362 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011830 con 0x7f7718162770 2024-01-16T14:54:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771cc9d700 1 -- 172.21.15.26:0/1910810362 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c011990 con 0x7f7718162770 2024-01-16T14:54:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771cc9d700 1 -- 172.21.15.26:0/1910810362 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c011d20 con 0x7f7718162770 2024-01-16T14:54:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1154003a20 con 0x7f11580c9fd0 2024-01-16T14:54:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080c8f40 con 0x7f1f08071390 2024-01-16T14:54:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.210+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080c5540 con 0x7f1f08071390 2024-01-16T14:54:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f114e7fc700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 0x7f113c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1154089040 con 0x7f11580c9fd0 2024-01-16T14:54:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f115db60700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 0x7f113c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee80052f0 con 0x7f1f08071390 2024-01-16T14:54:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771ff03700 1 -- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 msgr2=0x7f771806bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771ff03700 1 --2- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f771806bf60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f770c009840 tx=0x7f770c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771ff03700 1 -- 172.21.15.26:0/1910810362 shutdown_connections 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771ff03700 1 --2- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77180b19b0 0x7f7718162b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.212+0000 7f771ff03700 1 --2- 172.21.15.26:0/1910810362 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f771806bf60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f771ff03700 1 -- 172.21.15.26:0/1910810362 >> 172.21.15.26:0/1910810362 conn(0x7f77180b5070 msgr2=0x7f77180b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f771ff03700 1 -- 172.21.15.26:0/1910810362 shutdown_connections 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.213+0000 7f771ff03700 1 -- 172.21.15.26:0/1910810362 wait complete. 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.215+0000 7f771ff03700 1 Processor -- start 2024-01-16T14:54:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.209+0000 7f1586d9d700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 0x7f15881543b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f158000bdd0 tx=0x7f1580009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.215+0000 7f771ff03700 1 -- start start 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f771806c4a0 0x7f771806e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.215+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1f0002f430 con 0x7f1f08071390 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f1efdffb700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 0x7f1eec06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f0008b6a0 con 0x7f1f08071390 2024-01-16T14:54:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f77181fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718153220 con 0x7f77180b19b0 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181525a0 con 0x7f7718162770 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771ff03700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7718152880 con 0x7f771806c4a0 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.214+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bec054950 con 0x7f0bf4166600 2024-01-16T14:54:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f1f0cb83700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 0x7f1eec06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.216+0000 7f771e4a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f77181fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771d49e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771d49e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49826/0 (socket says 172.21.15.26:49826) 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771d49e700 1 -- 172.21.15.26:0/3696694319 learned_addr learned my addr 172.21.15.26:0/3696694319 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 -- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f771806c4a0 msgr2=0x7f771806e450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f771806c4a0 0x7f771806e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 -- 172.21.15.26:0/3696694319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 msgr2=0x7f771806e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 -- 172.21.15.26:0/3696694319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c0094a0 con 0x7f7718162770 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771d49e700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f771e4a0700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f77181fb6e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7708009d90 tx=0x7f770800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.217+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708003c20 con 0x7f7718162770 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.218+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770800f040 con 0x7f7718162770 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.218+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770800c930 con 0x7f7718162770 2024-01-16T14:54:54.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.218+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7718152b60 con 0x7f7718162770 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.218+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181fbc20 con 0x7f7718162770 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.219+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f770800ca90 con 0x7f7718162770 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.220+0000 7f77167fc700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 0x7f770406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.220+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7708089650 con 0x7f7718162770 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.220+0000 7f771dc9f700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 0x7f770406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.221+0000 7f771dc9f700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 0x7f770406d810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.221+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11400052f0 con 0x7f11580c9fd0 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.224+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f00056580 con 0x7f1f08071390 2024-01-16T14:54:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.224+0000 7f1f0cb83700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 0x7f1eec06d8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f081431a0 tx=0x7f1f04009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.224+0000 7f115db60700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 0x7f113c06d810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1148009ac0 tx=0x7f1148008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000cc10 con 0x7f15880ccc00 2024-01-16T14:54:54.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158000cd70 con 0x7f15880ccc00 2024-01-16T14:54:54.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580014680 con 0x7f15880ccc00 2024-01-16T14:54:54.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f80052f0 con 0x7f7718162770 2024-01-16T14:54:54.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880cf1b0 con 0x7f15880ccc00 2024-01-16T14:54:54.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.225+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15880cd4a0 con 0x7f15880ccc00 2024-01-16T14:54:54.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.226+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1154053f70 con 0x7f11580c9fd0 2024-01-16T14:54:54.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.226+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0052f0 con 0x7f15880ccc00 2024-01-16T14:54:54.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.231+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77080544c0 con 0x7f7718162770 2024-01-16T14:54:54.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.233+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f158001d3b0 con 0x7f15880ccc00 2024-01-16T14:54:54.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.233+0000 7f1576ffd700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 0x7f156806d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.234+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1580089f40 con 0x7f15880ccc00 2024-01-16T14:54:54.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.234+0000 7f158659c700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 0x7f156806d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.235+0000 7f158659c700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 0x7f156806d8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1578000f60 tx=0x7f1578007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.236+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1580054d90 con 0x7f15880ccc00 2024-01-16T14:54:54.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.273+0000 7f76f8bbf700 1 Processor -- start 2024-01-16T14:54:54.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.273+0000 7f76f8bbf700 1 -- start start 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76f4070a80 0x7f76f4070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f406cae0 0x7f76f40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f76f40ce980 0x7f76f416e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40c9fd0 con 0x7f76f4070a80 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40cab10 con 0x7f76f40ce980 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f8bbf700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40713a0 con 0x7f76f406cae0 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76f4070a80 0x7f76f4070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f76f4070a80 0x7f76f4070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49834/0 (socket says 172.21.15.26:49834) 2024-01-16T14:54:54.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.275+0000 7f76f259c700 1 -- 172.21.15.26:0/2250445224 learned_addr learned my addr 172.21.15.26:0/2250445224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.276+0000 7f76f2d9d700 1 --2- 172.21.15.26:0/2250445224 >> v2:172.21.15.93:3300/0 conn(0x7f76f40ce980 0x7f76f416e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.276+0000 7f76f1d9b700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f406cae0 0x7f76f40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 -- 172.21.15.26:0/2250445224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f406cae0 msgr2=0x7f76f40ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f406cae0 0x7f76f40ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 -- 172.21.15.26:0/2250445224 >> v2:172.21.15.93:3300/0 conn(0x7f76f40ce980 msgr2=0x7f76f416e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 --2- 172.21.15.26:0/2250445224 >> v2:172.21.15.93:3300/0 conn(0x7f76f40ce980 0x7f76f416e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 -- 172.21.15.26:0/2250445224 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f40cd7e0 con 0x7f76f4070a80 2024-01-16T14:54:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.278+0000 7f76f259c700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f4070a80 0x7f76f4070e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f40cc3b0 tx=0x7f76ec00acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21039f3e65bb26f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.279+0000 7f76f159a700 1 -- 172.21.15.26:0/2250445224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00a170 con 0x7f76f4070a80 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.279+0000 7f76f159a700 1 -- 172.21.15.26:0/2250445224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec00a2d0 con 0x7f76f4070a80 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.279+0000 7f76f159a700 1 -- 172.21.15.26:0/2250445224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec007930 con 0x7f76f4070a80 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2250445224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f4070a80 msgr2=0x7f76f4070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f4070a80 0x7f76f4070e60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76f40cc3b0 tx=0x7f76ec00acd0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2250445224 shutdown_connections 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2250445224 >> v2:172.21.15.93:3300/0 conn(0x7f76f40ce980 0x7f76f416e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f4070a80 0x7f76f4070e60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2250445224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f406cae0 0x7f76f40ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2250445224 >> 172.21.15.26:0/2250445224 conn(0x7f76f40cf980 msgr2=0x7f76f40cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.280+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2250445224 shutdown_connections 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.281+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2250445224 wait complete. 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.283+0000 7f76f8bbf700 1 Processor -- start 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.283+0000 7f76f8bbf700 1 -- start start 2024-01-16T14:54:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.284+0000 7f76f8bbf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76f406cae0 0x7f76f415be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.284+0000 7f76f8bbf700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f4070a80 0x7f76f40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f8bbf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f8bbf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40c99f0 con 0x7f76f40ce980 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f8bbf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40c9710 con 0x7f76f406cae0 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f8bbf700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f416a220 con 0x7f76f4070a80 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49842/0 (socket says 172.21.15.26:49842) 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.285+0000 7f76f2d9d700 1 -- 172.21.15.26:0/2848910014 learned_addr learned my addr 172.21.15.26:0/2848910014 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 -- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f4070a80 msgr2=0x7f76f40c8e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f4070a80 0x7f76f40c8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 -- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76f406cae0 msgr2=0x7f76f415be30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76f406cae0 0x7f76f415be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 -- 172.21.15.26:0/2848910014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec015c80 con 0x7f76f40ce980 2024-01-16T14:54:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.286+0000 7f76f2d9d700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76e4007e60 tx=0x7f76e400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.292+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4059620 con 0x7f76f40ce980 2024-01-16T14:54:54.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.292+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4059780 con 0x7f76f40ce980 2024-01-16T14:54:54.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.292+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4059b30 con 0x7f76f40ce980 2024-01-16T14:54:54.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.294+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f416a500 con 0x7f76f40ce980 2024-01-16T14:54:54.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.294+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f40c9350 con 0x7f76f40ce980 2024-01-16T14:54:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.298+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f76e4057070 con 0x7f76f40ce980 2024-01-16T14:54:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.304+0000 7f76e2ffd700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 0x7f76dc06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.304+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e40cd450 con 0x7f76f40ce980 2024-01-16T14:54:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.304+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f416a080 con 0x7f76f40ce980 2024-01-16T14:54:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.306+0000 7f76f259c700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 0x7f76dc06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.315+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76f416a080 con 0x7f76f40ce980 2024-01-16T14:54:54.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.349+0000 7f76f259c700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 0x7f76dc06da70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76ec000f80 tx=0x7f76ec01a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.559 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.574+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6b640055a0 con 0x7f6b84145870 2024-01-16T14:54:54.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.575+0000 7f6b6bfff700 1 -- 172.21.15.26:0/986737536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b74021020 con 0x7f6b84145870 2024-01-16T14:54:54.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.579+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 msgr2=0x7f6b7006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.579+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 0x7f6b7006d8e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6b840b5970 tx=0x7f6b78010000 comp rx=0 tx=0).stop 2024-01-16T14:54:54.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.579+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 msgr2=0x7f6b8415b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.579+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400e7e0 tx=0x7f6b7400eba0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 shutdown_connections 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b84166430 0x7f6b84132ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6b84070a80 0x7f6b84144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6b7006b420 0x7f6b7006d8e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 --2- 172.21.15.26:0/986737536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b84145870 0x7f6b8415b840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 >> 172.21.15.26:0/986737536 conn(0x7f6b840cf980 msgr2=0x7f6b840ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 shutdown_connections 2024-01-16T14:54:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.583+0000 7f6b89f34700 1 -- 172.21.15.26:0/986737536 wait complete. 2024-01-16T14:54:54.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-01-16T14:54:54.587 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.643+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f156c005870 con 0x7f15880ccc00 2024-01-16T14:54:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.643+0000 7f1576ffd700 1 -- 172.21.15.26:0/70776903 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1580019020 con 0x7f15880ccc00 2024-01-16T14:54:54.627 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-01-16T14:54:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 msgr2=0x7f156806d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 0x7f156806d8e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1578000f60 tx=0x7f1578007a40 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 msgr2=0x7f15881543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 0x7f15881543b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f158000bdd0 tx=0x7f1580009ed0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 shutdown_connections 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15880ccc00 0x7f15881543b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f158806da60 0x7f15881fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f156806b420 0x7f156806d8e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 --2- 172.21.15.26:0/70776903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15880af720 0x7f15881e5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 >> 172.21.15.26:0/70776903 conn(0x7f15880717f0 msgr2=0x7f15881e2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.646+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 shutdown_connections 2024-01-16T14:54:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.647+0000 7f158cc06700 1 -- 172.21.15.26:0/70776903 wait complete. 2024-01-16T14:54:54.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.650+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1ee8005870 con 0x7f1f08071390 2024-01-16T14:54:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.652+0000 7f1efdffb700 1 -- 172.21.15.26:0/974987830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f00017020 con 0x7f1f08071390 2024-01-16T14:54:54.635 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.655+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 msgr2=0x7f1eec06d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.655+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 0x7f1eec06d8e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f081431a0 tx=0x7f1f04009190 comp rx=0 tx=0).stop 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.655+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 msgr2=0x7f1f08159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.655+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 0x7f1f08159640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1f00014000 tx=0x7f1f00004ad0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 shutdown_connections 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0815b170 0x7f1f080c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1eec06b420 0x7f1eec06d8e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f08071390 0x7f1f08159640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 --2- 172.21.15.26:0/974987830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f080d04a0 0x7f1f080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 >> 172.21.15.26:0/974987830 conn(0x7f1f080cf980 msgr2=0x7f1f0806dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 shutdown_connections 2024-01-16T14:54:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.656+0000 7f1f0f5e8700 1 -- 172.21.15.26:0/974987830 wait complete. 2024-01-16T14:54:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.657+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1140005870 con 0x7f11580c9fd0 2024-01-16T14:54:54.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.658+0000 7f114e7fc700 1 -- 172.21.15.26:0/4070430716 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1154017020 con 0x7f11580c9fd0 2024-01-16T14:54:54.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.658+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0bf4075480 con 0x7f0bf4166600 2024-01-16T14:54:54.641 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.658+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb5740055a0 con 0x7fb59415a520 2024-01-16T14:54:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.659+0000 7fb57bfff700 1 -- 172.21.15.26:0/2615956198 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb588058400 con 0x7fb59415a520 2024-01-16T14:54:54.642 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.659+0000 7f0bf0ff9700 1 -- 172.21.15.26:0/3481388082 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0bec019090 con 0x7f0bf4166600 2024-01-16T14:54:54.644 INFO:teuthology.orchestra.run.smithi026.stdout:55834574853 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.662+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 msgr2=0x7f113c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.662+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 0x7f113c06d810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1148009ac0 tx=0x7f1148008040 comp rx=0 tx=0).stop 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.662+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 msgr2=0x7f11580b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.662+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 0x7f11580b34e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f115400c450 tx=0x7f115400c810 comp rx=0 tx=0).stop 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 shutdown_connections 2024-01-16T14:54:54.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f113c06b350 0x7f113c06d810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11580cef70 0x7f1158142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f115806cae0 0x7f11580b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 --2- 172.21.15.26:0/4070430716 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11580c9fd0 0x7f11580b34e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.663+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 >> 172.21.15.26:0/4070430716 conn(0x7f11580cf980 msgr2=0x7f1158156750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 msgr2=0x7fb57c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 0x7fb57c06da80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb594069500 tx=0x7fb584010000 comp rx=0 tx=0).stop 2024-01-16T14:54:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 msgr2=0x7fb59413c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb58800d800 tx=0x7fb58800dbc0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-01-16T14:54:54.648 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 msgr2=0x7f0be006d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 0x7f0be006d8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009c40 tx=0x7f0be4008030 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 msgr2=0x7f0bf41ee250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00b990 tx=0x7f0bec00bca0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 shutdown_connections 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0be006b420 0x7f0be006d8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bf4166600 0x7f0bf41ee250 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0bf4070a80 0x7f0bf4137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 --2- 172.21.15.26:0/3481388082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0bf4145870 0x7f0bf4165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.664+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 >> 172.21.15.26:0/3481388082 conn(0x7f0bf40cf980 msgr2=0x7f0bf40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 shutdown_connections 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb59413e690 0x7fb59415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb59415a520 0x7fb59413c360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb57c06b5c0 0x7fb57c06da80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 shutdown_connections 2024-01-16T14:54:54.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7f115fdc4700 1 -- 172.21.15.26:0/4070430716 wait complete. 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 --2- 172.21.15.26:0/2615956198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415db10 0x7fb59413e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 >> 172.21.15.26:0/2615956198 conn(0x7fb5940cf980 msgr2=0x7fb5940af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 shutdown_connections 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7fb599a12700 1 -- 172.21.15.26:0/2615956198 wait complete. 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 shutdown_connections 2024-01-16T14:54:54.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.665+0000 7f0bfa62c700 1 -- 172.21.15.26:0/3481388082 wait complete. 2024-01-16T14:54:54.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.669+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f76f8005870 con 0x7f7718162770 2024-01-16T14:54:54.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.670+0000 7f77167fc700 1 -- 172.21.15.26:0/3696694319 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7708057ae0 con 0x7f7718162770 2024-01-16T14:54:54.654 INFO:teuthology.orchestra.run.smithi026.stdout:55834574855 2024-01-16T14:54:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 msgr2=0x7f770406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 0x7f770406d810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f770c0090b0 tx=0x7f770c008030 comp rx=0 tx=0).stop 2024-01-16T14:54:54.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 msgr2=0x7f77181fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f77181fb6e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7708009d90 tx=0x7f770800c3a0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 shutdown_connections 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7718162770 0x7f77181fb6e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f771806c4a0 0x7f771806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770406b350 0x7f770406d810 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 --2- 172.21.15.26:0/3696694319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77180b19b0 0x7f771806e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 >> 172.21.15.26:0/3696694319 conn(0x7f77180b5070 msgr2=0x7f7718161a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.674+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 shutdown_connections 2024-01-16T14:54:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.675+0000 7f771ff03700 1 -- 172.21.15.26:0/3696694319 wait complete. 2024-01-16T14:54:54.663 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-01-16T14:54:54.663 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-01-16T14:54:54.673 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-01-16T14:54:54.676 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-01-16T14:54:54.676 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-01-16T14:54:54.680 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.723+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f76f416aa20 con 0x7f76f40ce980 2024-01-16T14:54:54.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.723+0000 7f76e2ffd700 1 -- 172.21.15.26:0/2848910014 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76f416aa20 con 0x7f76f40ce980 2024-01-16T14:54:54.706 INFO:teuthology.orchestra.run.smithi026.stdout:55834574854 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 msgr2=0x7f76dc06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 0x7f76dc06da70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76ec000f80 tx=0x7f76ec01a000 comp rx=0 tx=0).stop 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 msgr2=0x7f76f416cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76e4007e60 tx=0x7f76e400db40 comp rx=0 tx=0).stop 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 shutdown_connections 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f76dc06b5b0 0x7f76dc06da70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76f40ce980 0x7f76f416cf40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76f406cae0 0x7f76f415be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 --2- 172.21.15.26:0/2848910014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76f4070a80 0x7f76f40c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 >> 172.21.15.26:0/2848910014 conn(0x7f76f40cf980 msgr2=0x7f76f40b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 shutdown_connections 2024-01-16T14:54:54.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.726+0000 7f76f8bbf700 1 -- 172.21.15.26:0/2848910014 wait complete. 2024-01-16T14:54:54.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-01-16T14:54:54.723 DEBUG:teuthology.parallel:result is None 2024-01-16T14:54:54.723 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-16T14:54:54.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:54:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.920+0000 7f9a3466e700 1 Processor -- start 2024-01-16T14:54:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.920+0000 7f9a3466e700 1 -- start start 2024-01-16T14:54:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3466e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 0x7f9a2c072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3466e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a2c0b3d90 0x7f9a2c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3466e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2c15f3e0 con 0x7f9a2c06bac0 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3466e700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2c0affd0 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3466e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c073500 con 0x7f9a2c0726a0 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31c09700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a2c0b3d90 0x7f9a2c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a32c0b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a2c06bac0 0x7f9a2c06bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58566/0 (socket says 172.21.15.26:58566) 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a32c0b700 1 -- 172.21.15.26:0/3212743629 learned_addr learned my addr 172.21.15.26:0/3212743629 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a3240a700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 0x7f9a2c072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31c09700 1 -- 172.21.15.26:0/3212743629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 msgr2=0x7f9a2c072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31c09700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 0x7f9a2c072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31c09700 1 -- 172.21.15.26:0/3212743629 >> v1:172.21.15.26:6789/0 conn(0x7f9a2c06bac0 legacy=0x7f9a2c06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31c09700 1 -- 172.21.15.26:0/3212743629 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c15e7d0 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.921+0000 7f9a31408700 1 -- 172.21.15.26:0/3212743629 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755150712 0 0) 0x7f9a2c15f3e0 con 0x7f9a2c06bac0 2024-01-16T14:54:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a3240a700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 0x7f9a2c072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a31c09700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c0b3d90 0x7f9a2c072fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009840 tx=0x7f9a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17cc95be91faff7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a31408700 1 -- 172.21.15.26:0/3212743629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c011830 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a31408700 1 -- 172.21.15.26:0/3212743629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c011990 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a31408700 1 -- 172.21.15.26:0/3212743629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c011d20 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a3466e700 1 -- 172.21.15.26:0/3212743629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c0b3d90 msgr2=0x7f9a2c072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.922+0000 7f9a3466e700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c0b3d90 0x7f9a2c072fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009840 tx=0x7f9a1c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 -- 172.21.15.26:0/3212743629 shutdown_connections 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0726a0 0x7f9a2c072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 --2- 172.21.15.26:0/3212743629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c0b3d90 0x7f9a2c072fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 -- 172.21.15.26:0/3212743629 >> 172.21.15.26:0/3212743629 conn(0x7f9a2c0717f0 msgr2=0x7f9a2c0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 -- 172.21.15.26:0/3212743629 shutdown_connections 2024-01-16T14:54:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.923+0000 7f9a3466e700 1 -- 172.21.15.26:0/3212743629 wait complete. 2024-01-16T14:54:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.924+0000 7f9a3466e700 1 Processor -- start 2024-01-16T14:54:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.924+0000 7f9a3466e700 1 -- start start 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a2c0726a0 0x7f9a2c1443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0b3d90 0x7f9a2c1448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c142960 con 0x7f9a2c06bac0 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c144f10 con 0x7f9a2c0726a0 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3466e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1451f0 con 0x7f9a2c0b3d90 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.925+0000 7f9a3240a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55336/0 (socket says 172.21.15.26:55336) 2024-01-16T14:54:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 -- 172.21.15.26:0/879125383 learned_addr learned my addr 172.21.15.26:0/879125383 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a31c09700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a2c0726a0 0x7f9a2c1443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a32c0b700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0b3d90 0x7f9a2c1448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 -- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0b3d90 msgr2=0x7f9a2c1448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0b3d90 0x7f9a2c1448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 -- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a2c0726a0 msgr2=0x7f9a2c1443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a2c0726a0 0x7f9a2c1443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 -- 172.21.15.26:0/879125383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0094a0 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3240a700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800e810 tx=0x7f9a2800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800c070 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28004230 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c142310 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280036a0 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.926+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c14fbc0 con 0x7f9a2c06bac0 2024-01-16T14:54:54.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.927+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2c075480 con 0x7f9a2c06bac0 2024-01-16T14:54:54.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.928+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9a28003800 con 0x7f9a2c06bac0 2024-01-16T14:54:54.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.928+0000 7f9a22ffd700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 0x7f9a1006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:54.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.928+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a28089810 con 0x7f9a2c06bac0 2024-01-16T14:54:54.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.928+0000 7f9a31c09700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 0x7f9a1006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:54.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.929+0000 7f9a31c09700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 0x7f9a1006d810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009840 tx=0x7f9a1c01d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:54.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:54.933+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a28054680 con 0x7f9a2c06bac0 2024-01-16T14:54:55.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.138+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a2c0b06a0 con 0x7f9a1006b350 2024-01-16T14:54:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.146+0000 7f9a22ffd700 1 -- 172.21.15.26:0/879125383 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264550 (secure 0 0 0) 0x7f9a2c0b06a0 con 0x7f9a1006b350 2024-01-16T14:54:55.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 msgr2=0x7f9a1006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 0x7f9a1006d810 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009840 tx=0x7f9a1c01d040 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 msgr2=0x7f9a2c15c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800e810 tx=0x7f9a2800eb20 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 shutdown_connections 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a2c0b3d90 0x7f9a2c1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9a1006b350 0x7f9a1006d810 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2c06bac0 0x7f9a2c15c790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 --2- 172.21.15.26:0/879125383 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a2c0726a0 0x7f9a2c1443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 >> 172.21.15.26:0/879125383 conn(0x7f9a2c0717f0 msgr2=0x7f9a2c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:55.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 shutdown_connections 2024-01-16T14:54:55.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.149+0000 7f9a3466e700 1 -- 172.21.15.26:0/879125383 wait complete. 2024-01-16T14:54:55.135 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:54:55.146 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-01-16T14:54:53.351709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2973696,"data_stored":1787728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854111+0000","last_change":"2024-01-16T14:54:45.877716+0000","last_active":"2024-01-16T14:54:46.854111+0000","last_peered":"2024-01-16T14:54:46.854111+0000","last_clean":"2024-01-16T14:54:46.854111+0000","last_became_active":"2024-01-16T14:54:45.876462+0000","last_became_peered":"2024-01-16T14:54:45.876462+0000","last_unstale":"2024-01-16T14:54:46.854111+0000","last_undegraded":"2024-01-16T14:54:46.854111+0000","last_fullsized":"2024-01-16T14:54:46.854111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262923+0000","last_change":"2024-01-16T14:54:45.880623+0000","last_active":"2024-01-16T14:54:47.262923+0000","last_peered":"2024-01-16T14:54:47.262923+0000","last_clean":"2024-01-16T14:54:47.262923+0000","last_became_active":"2024-01-16T14:54:45.878584+0000","last_became_peered":"2024-01-16T14:54:45.878584+0000","last_unstale":"2024-01-16T14:54:47.262923+0000","last_undegraded":"2024-01-16T14:54:47.262923+0000","last_fullsized":"2024-01-16T14:54:47.262923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261939+0000","last_change":"2024-01-16T14:54:45.875065+0000","last_active":"2024-01-16T14:54:47.261939+0000","last_peered":"2024-01-16T14:54:47.261939+0000","last_clean":"2024-01-16T14:54:47.261939+0000","last_became_active":"2024-01-16T14:54:45.873809+0000","last_became_peered":"2024-01-16T14:54:45.873809+0000","last_unstale":"2024-01-16T14:54:47.261939+0000","last_undegraded":"2024-01-16T14:54:47.261939+0000","last_fullsized":"2024-01-16T14:54:47.261939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261991+0000","last_change":"2024-01-16T14:54:45.877233+0000","last_active":"2024-01-16T14:54:47.261991+0000","last_peered":"2024-01-16T14:54:47.261991+0000","last_clean":"2024-01-16T14:54:47.261991+0000","last_became_active":"2024-01-16T14:54:45.876940+0000","last_became_peered":"2024-01-16T14:54:45.876940+0000","last_unstale":"2024-01-16T14:54:47.261991+0000","last_undegraded":"2024-01-16T14:54:47.261991+0000","last_fullsized":"2024-01-16T14:54:47.261991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853429+0000","last_change":"2024-01-16T14:54:45.874709+0000","last_active":"2024-01-16T14:54:46.853429+0000","last_peered":"2024-01-16T14:54:46.853429+0000","last_clean":"2024-01-16T14:54:46.853429+0000","last_became_active":"2024-01-16T14:54:45.872900+0000","last_became_peered":"2024-01-16T14:54:45.872900+0000","last_unstale":"2024-01-16T14:54:46.853429+0000","last_undegraded":"2024-01-16T14:54:46.853429+0000","last_fullsized":"2024-01-16T14:54:46.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261215+0000","last_change":"2024-01-16T14:54:45.882475+0000","last_active":"2024-01-16T14:54:47.261215+0000","last_peered":"2024-01-16T14:54:47.261215+0000","last_clean":"2024-01-16T14:54:47.261215+0000","last_became_active":"2024-01-16T14:54:45.882150+0000","last_became_peered":"2024-01-16T14:54:45.882150+0000","last_unstale":"2024-01-16T14:54:47.261215+0000","last_undegraded":"2024-01-16T14:54:47.261215+0000","last_fullsized":"2024-01-16T14:54:47.261215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853589+0000","last_change":"2024-01-16T14:54:45.875750+0000","last_active":"2024-01-16T14:54:46.853589+0000","last_peered":"2024-01-16T14:54:46.853589+0000","last_clean":"2024-01-16T14:54:46.853589+0000","last_became_active":"2024-01-16T14:54:45.875474+0000","last_became_peered":"2024-01-16T14:54:45.875474+0000","last_unstale":"2024-01-16T14:54:46.853589+0000","last_undegraded":"2024-01-16T14:54:46.853589+0000","last_fullsized":"2024-01-16T14:54:46.853589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853586+0000","last_change":"2024-01-16T14:54:45.882976+0000","last_active":"2024-01-16T14:54:46.853586+0000","last_peered":"2024-01-16T14:54:46.853586+0000","last_clean":"2024-01-16T14:54:46.853586+0000","last_became_active":"2024-01-16T14:54:45.882628+0000","last_became_peered":"2024-01-16T14:54:45.882628+0000","last_unstale":"2024-01-16T14:54:46.853586+0000","last_undegraded":"2024-01-16T14:54:46.853586+0000","last_fullsized":"2024-01-16T14:54:46.853586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853730+0000","last_change":"2024-01-16T14:54:45.885967+0000","last_active":"2024-01-16T14:54:46.853730+0000","last_peered":"2024-01-16T14:54:46.853730+0000","last_clean":"2024-01-16T14:54:46.853730+0000","last_became_active":"2024-01-16T14:54:45.885700+0000","last_became_peered":"2024-01-16T14:54:45.885700+0000","last_unstale":"2024-01-16T14:54:46.853730+0000","last_undegraded":"2024-01-16T14:54:46.853730+0000","last_fullsized":"2024-01-16T14:54:46.853730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262320+0000","last_change":"2024-01-16T14:54:45.878748+0000","last_active":"2024-01-16T14:54:47.262320+0000","last_peered":"2024-01-16T14:54:47.262320+0000","last_clean":"2024-01-16T14:54:47.262320+0000","last_became_active":"2024-01-16T14:54:45.878446+0000","last_became_peered":"2024-01-16T14:54:45.878446+0000","last_unstale":"2024-01-16T14:54:47.262320+0000","last_undegraded":"2024-01-16T14:54:47.262320+0000","last_fullsized":"2024-01-16T14:54:47.262320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261357+0000","last_change":"2024-01-16T14:54:45.879232+0000","last_active":"2024-01-16T14:54:47.261357+0000","last_peered":"2024-01-16T14:54:47.261357+0000","last_clean":"2024-01-16T14:54:47.261357+0000","last_became_active":"2024-01-16T14:54:45.878875+0000","last_became_peered":"2024-01-16T14:54:45.878875+0000","last_unstale":"2024-01-16T14:54:47.261357+0000","last_undegraded":"2024-01-16T14:54:47.261357+0000","last_fullsized":"2024-01-16T14:54:47.261357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855341+0000","last_change":"2024-01-16T14:54:45.862373+0000","last_active":"2024-01-16T14:54:46.855341+0000","last_peered":"2024-01-16T14:54:46.855341+0000","last_clean":"2024-01-16T14:54:46.855341+0000","last_became_active":"2024-01-16T14:54:45.862110+0000","last_became_peered":"2024-01-16T14:54:45.862110+0000","last_unstale":"2024-01-16T14:54:46.855341+0000","last_undegraded":"2024-01-16T14:54:46.855341+0000","last_fullsized":"2024-01-16T14:54:46.855341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854063+0000","last_change":"2024-01-16T14:54:45.867064+0000","last_active":"2024-01-16T14:54:46.854063+0000","last_peered":"2024-01-16T14:54:46.854063+0000","last_clean":"2024-01-16T14:54:46.854063+0000","last_became_active":"2024-01-16T14:54:45.866257+0000","last_became_peered":"2024-01-16T14:54:45.866257+0000","last_unstale":"2024-01-16T14:54:46.854063+0000","last_undegraded":"2024-01-16T14:54:46.854063+0000","last_fullsized":"2024-01-16T14:54:46.854063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261852+0000","last_change":"2024-01-16T14:54:45.875851+0000","last_active":"2024-01-16T14:54:47.261852+0000","last_peered":"2024-01-16T14:54:47.261852+0000","last_clean":"2024-01-16T14:54:47.261852+0000","last_became_active":"2024-01-16T14:54:45.875202+0000","last_became_peered":"2024-01-16T14:54:45.875202+0000","last_unstale":"2024-01-16T14:54:47.261852+0000","last_undegraded":"2024-01-16T14:54:47.261852+0000","last_fullsized":"2024-01-16T14:54:47.261852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:35.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261833+0000","last_change":"2024-01-16T14:54:45.858130+0000","last_active":"2024-01-16T14:54:47.261833+0000","last_peered":"2024-01-16T14:54:47.261833+0000","last_clean":"2024-01-16T14:54:47.261833+0000","last_became_active":"2024-01-16T14:54:45.856812+0000","last_became_peered":"2024-01-16T14:54:45.856812+0000","last_unstale":"2024-01-16T14:54:47.261833+0000","last_undegraded":"2024-01-16T14:54:47.261833+0000","last_fullsized":"2024-01-16T14:54:47.261833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261530+0000","last_change":"2024-01-16T14:54:45.883516+0000","last_active":"2024-01-16T14:54:47.261530+0000","last_peered":"2024-01-16T14:54:47.261530+0000","last_clean":"2024-01-16T14:54:47.261530+0000","last_became_active":"2024-01-16T14:54:45.883259+0000","last_became_peered":"2024-01-16T14:54:45.883259+0000","last_unstale":"2024-01-16T14:54:47.261530+0000","last_undegraded":"2024-01-16T14:54:47.261530+0000","last_fullsized":"2024-01-16T14:54:47.261530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262193+0000","last_change":"2024-01-16T14:54:45.883796+0000","last_active":"2024-01-16T14:54:47.262193+0000","last_peered":"2024-01-16T14:54:47.262193+0000","last_clean":"2024-01-16T14:54:47.262193+0000","last_became_active":"2024-01-16T14:54:45.883558+0000","last_became_peered":"2024-01-16T14:54:45.883558+0000","last_unstale":"2024-01-16T14:54:47.262193+0000","last_undegraded":"2024-01-16T14:54:47.262193+0000","last_fullsized":"2024-01-16T14:54:47.262193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262894+0000","last_change":"2024-01-16T14:54:45.867763+0000","last_active":"2024-01-16T14:54:47.262894+0000","last_peered":"2024-01-16T14:54:47.262894+0000","last_clean":"2024-01-16T14:54:47.262894+0000","last_became_active":"2024-01-16T14:54:45.867441+0000","last_became_peered":"2024-01-16T14:54:45.867441+0000","last_unstale":"2024-01-16T14:54:47.262894+0000","last_undegraded":"2024-01-16T14:54:47.262894+0000","last_fullsized":"2024-01-16T14:54:47.262894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853971+0000","last_change":"2024-01-16T14:54:45.873151+0000","last_active":"2024-01-16T14:54:46.853971+0000","last_peered":"2024-01-16T14:54:46.853971+0000","last_clean":"2024-01-16T14:54:46.853971+0000","last_became_active":"2024-01-16T14:54:45.872841+0000","last_became_peered":"2024-01-16T14:54:45.872841+0000","last_unstale":"2024-01-16T14:54:46.853971+0000","last_undegraded":"2024-01-16T14:54:46.853971+0000","last_fullsized":"2024-01-16T14:54:46.853971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262224+0000","last_change":"2024-01-16T14:54:44.860003+0000","last_active":"2024-01-16T14:54:47.262224+0000","last_peered":"2024-01-16T14:54:47.262224+0000","last_clean":"2024-01-16T14:54:47.262224+0000","last_became_active":"2024-01-16T14:54:44.859774+0000","last_became_peered":"2024-01-16T14:54:44.859774+0000","last_unstale":"2024-01-16T14:54:47.262224+0000","last_undegraded":"2024-01-16T14:54:47.262224+0000","last_fullsized":"2024-01-16T14:54:47.262224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854240+0000","last_change":"2024-01-16T14:54:45.867265+0000","last_active":"2024-01-16T14:54:46.854240+0000","last_peered":"2024-01-16T14:54:46.854240+0000","last_clean":"2024-01-16T14:54:46.854240+0000","last_became_active":"2024-01-16T14:54:45.865756+0000","last_became_peered":"2024-01-16T14:54:45.865756+0000","last_unstale":"2024-01-16T14:54:46.854240+0000","last_undegraded":"2024-01-16T14:54:46.854240+0000","last_fullsized":"2024-01-16T14:54:46.854240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854088+0000","last_change":"2024-01-16T14:54:44.853662+0000","last_active":"2024-01-16T14:54:46.854088+0000","last_peered":"2024-01-16T14:54:46.854088+0000","last_clean":"2024-01-16T14:54:46.854088+0000","last_became_active":"2024-01-16T14:54:44.853379+0000","last_became_peered":"2024-01-16T14:54:44.853379+0000","last_unstale":"2024-01-16T14:54:46.854088+0000","last_undegraded":"2024-01-16T14:54:46.854088+0000","last_fullsized":"2024-01-16T14:54:46.854088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262160+0000","last_change":"2024-01-16T14:54:45.880017+0000","last_active":"2024-01-16T14:54:47.262160+0000","last_peered":"2024-01-16T14:54:47.262160+0000","last_clean":"2024-01-16T14:54:47.262160+0000","last_became_active":"2024-01-16T14:54:45.879369+0000","last_became_peered":"2024-01-16T14:54:45.879369+0000","last_unstale":"2024-01-16T14:54:47.262160+0000","last_undegraded":"2024-01-16T14:54:47.262160+0000","last_fullsized":"2024-01-16T14:54:47.262160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853963+0000","last_change":"2024-01-16T14:54:44.852663+0000","last_active":"2024-01-16T14:54:46.853963+0000","last_peered":"2024-01-16T14:54:46.853963+0000","last_clean":"2024-01-16T14:54:46.853963+0000","last_became_active":"2024-01-16T14:54:44.852346+0000","last_became_peered":"2024-01-16T14:54:44.852346+0000","last_unstale":"2024-01-16T14:54:46.853963+0000","last_undegraded":"2024-01-16T14:54:46.853963+0000","last_fullsized":"2024-01-16T14:54:46.853963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:54.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261811+0000","last_change":"2024-01-16T14:54:45.886559+0000","last_active":"2024-01-16T14:54:47.261811+0000","last_peered":"2024-01-16T14:54:47.261811+0000","last_clean":"2024-01-16T14:54:47.261811+0000","last_became_active":"2024-01-16T14:54:45.886343+0000","last_became_peered":"2024-01-16T14:54:45.886343+0000","last_unstale":"2024-01-16T14:54:47.261811+0000","last_undegraded":"2024-01-16T14:54:47.261811+0000","last_fullsized":"2024-01-16T14:54:47.261811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853331+0000","last_change":"2024-01-16T14:54:44.852311+0000","last_active":"2024-01-16T14:54:46.853331+0000","last_peered":"2024-01-16T14:54:46.853331+0000","last_clean":"2024-01-16T14:54:46.853331+0000","last_became_active":"2024-01-16T14:54:44.852018+0000","last_became_peered":"2024-01-16T14:54:44.852018+0000","last_unstale":"2024-01-16T14:54:46.853331+0000","last_undegraded":"2024-01-16T14:54:46.853331+0000","last_fullsized":"2024-01-16T14:54:46.853331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262110+0000","last_change":"2024-01-16T14:54:45.883384+0000","last_active":"2024-01-16T14:54:47.262110+0000","last_peered":"2024-01-16T14:54:47.262110+0000","last_clean":"2024-01-16T14:54:47.262110+0000","last_became_active":"2024-01-16T14:54:45.883094+0000","last_became_peered":"2024-01-16T14:54:45.883094+0000","last_unstale":"2024-01-16T14:54:47.262110+0000","last_undegraded":"2024-01-16T14:54:47.262110+0000","last_fullsized":"2024-01-16T14:54:47.262110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853863+0000","last_change":"2024-01-16T14:54:44.846655+0000","last_active":"2024-01-16T14:54:46.853863+0000","last_peered":"2024-01-16T14:54:46.853863+0000","last_clean":"2024-01-16T14:54:46.853863+0000","last_became_active":"2024-01-16T14:54:44.846359+0000","last_became_peered":"2024-01-16T14:54:44.846359+0000","last_unstale":"2024-01-16T14:54:46.853863+0000","last_undegraded":"2024-01-16T14:54:46.853863+0000","last_fullsized":"2024-01-16T14:54:46.853863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:28.971339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854823+0000","last_change":"2024-01-16T14:54:45.857956+0000","last_active":"2024-01-16T14:54:46.854823+0000","last_peered":"2024-01-16T14:54:46.854823+0000","last_clean":"2024-01-16T14:54:46.854823+0000","last_became_active":"2024-01-16T14:54:45.857645+0000","last_became_peered":"2024-01-16T14:54:45.857645+0000","last_unstale":"2024-01-16T14:54:46.854823+0000","last_undegraded":"2024-01-16T14:54:46.854823+0000","last_fullsized":"2024-01-16T14:54:46.854823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:39.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261584+0000","last_change":"2024-01-16T14:54:44.851910+0000","last_active":"2024-01-16T14:54:47.261584+0000","last_peered":"2024-01-16T14:54:47.261584+0000","last_clean":"2024-01-16T14:54:47.261584+0000","last_became_active":"2024-01-16T14:54:44.851631+0000","last_became_peered":"2024-01-16T14:54:44.851631+0000","last_unstale":"2024-01-16T14:54:47.261584+0000","last_undegraded":"2024-01-16T14:54:47.261584+0000","last_fullsized":"2024-01-16T14:54:47.261584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:17.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262365+0000","last_change":"2024-01-16T14:54:45.875708+0000","last_active":"2024-01-16T14:54:47.262365+0000","last_peered":"2024-01-16T14:54:47.262365+0000","last_clean":"2024-01-16T14:54:47.262365+0000","last_became_active":"2024-01-16T14:54:45.875328+0000","last_became_peered":"2024-01-16T14:54:45.875328+0000","last_unstale":"2024-01-16T14:54:47.262365+0000","last_undegraded":"2024-01-16T14:54:47.262365+0000","last_fullsized":"2024-01-16T14:54:47.262365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853648+0000","last_change":"2024-01-16T14:54:44.855286+0000","last_active":"2024-01-16T14:54:46.853648+0000","last_peered":"2024-01-16T14:54:46.853648+0000","last_clean":"2024-01-16T14:54:46.853648+0000","last_became_active":"2024-01-16T14:54:44.854465+0000","last_became_peered":"2024-01-16T14:54:44.854465+0000","last_unstale":"2024-01-16T14:54:46.853648+0000","last_undegraded":"2024-01-16T14:54:46.853648+0000","last_fullsized":"2024-01-16T14:54:46.853648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853759+0000","last_change":"2024-01-16T14:54:45.868676+0000","last_active":"2024-01-16T14:54:46.853759+0000","last_peered":"2024-01-16T14:54:46.853759+0000","last_clean":"2024-01-16T14:54:46.853759+0000","last_became_active":"2024-01-16T14:54:45.868354+0000","last_became_peered":"2024-01-16T14:54:45.868354+0000","last_unstale":"2024-01-16T14:54:46.853759+0000","last_undegraded":"2024-01-16T14:54:46.853759+0000","last_fullsized":"2024-01-16T14:54:46.853759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261155+0000","last_change":"2024-01-16T14:54:44.855323+0000","last_active":"2024-01-16T14:54:47.261155+0000","last_peered":"2024-01-16T14:54:47.261155+0000","last_clean":"2024-01-16T14:54:47.261155+0000","last_became_active":"2024-01-16T14:54:44.855035+0000","last_became_peered":"2024-01-16T14:54:44.855035+0000","last_unstale":"2024-01-16T14:54:47.261155+0000","last_undegraded":"2024-01-16T14:54:47.261155+0000","last_fullsized":"2024-01-16T14:54:47.261155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:33.068253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262370+0000","last_change":"2024-01-16T14:54:45.864396+0000","last_active":"2024-01-16T14:54:47.262370+0000","last_peered":"2024-01-16T14:54:47.262370+0000","last_clean":"2024-01-16T14:54:47.262370+0000","last_became_active":"2024-01-16T14:54:45.864052+0000","last_became_peered":"2024-01-16T14:54:45.864052+0000","last_unstale":"2024-01-16T14:54:47.262370+0000","last_undegraded":"2024-01-16T14:54:47.262370+0000","last_fullsized":"2024-01-16T14:54:47.262370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262332+0000","last_change":"2024-01-16T14:54:44.857192+0000","last_active":"2024-01-16T14:54:47.262332+0000","last_peered":"2024-01-16T14:54:47.262332+0000","last_clean":"2024-01-16T14:54:47.262332+0000","last_became_active":"2024-01-16T14:54:44.856880+0000","last_became_peered":"2024-01-16T14:54:44.856880+0000","last_unstale":"2024-01-16T14:54:47.262332+0000","last_undegraded":"2024-01-16T14:54:47.262332+0000","last_fullsized":"2024-01-16T14:54:47.262332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261868+0000","last_change":"2024-01-16T14:54:45.884208+0000","last_active":"2024-01-16T14:54:47.261868+0000","last_peered":"2024-01-16T14:54:47.261868+0000","last_clean":"2024-01-16T14:54:47.261868+0000","last_became_active":"2024-01-16T14:54:45.882540+0000","last_became_peered":"2024-01-16T14:54:45.882540+0000","last_unstale":"2024-01-16T14:54:47.261868+0000","last_undegraded":"2024-01-16T14:54:47.261868+0000","last_fullsized":"2024-01-16T14:54:47.261868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261721+0000","last_change":"2024-01-16T14:54:44.854856+0000","last_active":"2024-01-16T14:54:47.261721+0000","last_peered":"2024-01-16T14:54:47.261721+0000","last_clean":"2024-01-16T14:54:47.261721+0000","last_became_active":"2024-01-16T14:54:44.854543+0000","last_became_peered":"2024-01-16T14:54:44.854543+0000","last_unstale":"2024-01-16T14:54:47.261721+0000","last_undegraded":"2024-01-16T14:54:47.261721+0000","last_fullsized":"2024-01-16T14:54:47.261721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262312+0000","last_change":"2024-01-16T14:54:45.880240+0000","last_active":"2024-01-16T14:54:47.262312+0000","last_peered":"2024-01-16T14:54:47.262312+0000","last_clean":"2024-01-16T14:54:47.262312+0000","last_became_active":"2024-01-16T14:54:45.879764+0000","last_became_peered":"2024-01-16T14:54:45.879764+0000","last_unstale":"2024-01-16T14:54:47.262312+0000","last_undegraded":"2024-01-16T14:54:47.262312+0000","last_fullsized":"2024-01-16T14:54:47.262312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262763+0000","last_change":"2024-01-16T14:54:44.855449+0000","last_active":"2024-01-16T14:54:47.262763+0000","last_peered":"2024-01-16T14:54:47.262763+0000","last_clean":"2024-01-16T14:54:47.262763+0000","last_became_active":"2024-01-16T14:54:44.855037+0000","last_became_peered":"2024-01-16T14:54:44.855037+0000","last_unstale":"2024-01-16T14:54:47.262763+0000","last_undegraded":"2024-01-16T14:54:47.262763+0000","last_fullsized":"2024-01-16T14:54:47.262763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855392+0000","last_change":"2024-01-16T14:54:45.870932+0000","last_active":"2024-01-16T14:54:46.855392+0000","last_peered":"2024-01-16T14:54:46.855392+0000","last_clean":"2024-01-16T14:54:46.855392+0000","last_became_active":"2024-01-16T14:54:45.870016+0000","last_became_peered":"2024-01-16T14:54:45.870016+0000","last_unstale":"2024-01-16T14:54:46.855392+0000","last_undegraded":"2024-01-16T14:54:46.855392+0000","last_fullsized":"2024-01-16T14:54:46.855392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261353+0000","last_change":"2024-01-16T14:54:44.854762+0000","last_active":"2024-01-16T14:54:47.261353+0000","last_peered":"2024-01-16T14:54:47.261353+0000","last_clean":"2024-01-16T14:54:47.261353+0000","last_became_active":"2024-01-16T14:54:44.854507+0000","last_became_peered":"2024-01-16T14:54:44.854507+0000","last_unstale":"2024-01-16T14:54:47.261353+0000","last_undegraded":"2024-01-16T14:54:47.261353+0000","last_fullsized":"2024-01-16T14:54:47.261353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853580+0000","last_change":"2024-01-16T14:54:45.879814+0000","last_active":"2024-01-16T14:54:46.853580+0000","last_peered":"2024-01-16T14:54:46.853580+0000","last_clean":"2024-01-16T14:54:46.853580+0000","last_became_active":"2024-01-16T14:54:45.873897+0000","last_became_peered":"2024-01-16T14:54:45.873897+0000","last_unstale":"2024-01-16T14:54:46.853580+0000","last_undegraded":"2024-01-16T14:54:46.853580+0000","last_fullsized":"2024-01-16T14:54:46.853580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262205+0000","last_change":"2024-01-16T14:54:44.860218+0000","last_active":"2024-01-16T14:54:47.262205+0000","last_peered":"2024-01-16T14:54:47.262205+0000","last_clean":"2024-01-16T14:54:47.262205+0000","last_became_active":"2024-01-16T14:54:44.859920+0000","last_became_peered":"2024-01-16T14:54:44.859920+0000","last_unstale":"2024-01-16T14:54:47.262205+0000","last_undegraded":"2024-01-16T14:54:47.262205+0000","last_fullsized":"2024-01-16T14:54:47.262205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262345+0000","last_change":"2024-01-16T14:54:45.875405+0000","last_active":"2024-01-16T14:54:47.262345+0000","last_peered":"2024-01-16T14:54:47.262345+0000","last_clean":"2024-01-16T14:54:47.262345+0000","last_became_active":"2024-01-16T14:54:45.874428+0000","last_became_peered":"2024-01-16T14:54:45.874428+0000","last_unstale":"2024-01-16T14:54:47.262345+0000","last_undegraded":"2024-01-16T14:54:47.262345+0000","last_fullsized":"2024-01-16T14:54:47.262345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:27.912278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262076+0000","last_change":"2024-01-16T14:54:44.857601+0000","last_active":"2024-01-16T14:54:47.262076+0000","last_peered":"2024-01-16T14:54:47.262076+0000","last_clean":"2024-01-16T14:54:47.262076+0000","last_became_active":"2024-01-16T14:54:44.857282+0000","last_became_peered":"2024-01-16T14:54:44.857282+0000","last_unstale":"2024-01-16T14:54:47.262076+0000","last_undegraded":"2024-01-16T14:54:47.262076+0000","last_fullsized":"2024-01-16T14:54:47.262076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853602+0000","last_change":"2024-01-16T14:54:45.873626+0000","last_active":"2024-01-16T14:54:46.853602+0000","last_peered":"2024-01-16T14:54:46.853602+0000","last_clean":"2024-01-16T14:54:46.853602+0000","last_became_active":"2024-01-16T14:54:45.872632+0000","last_became_peered":"2024-01-16T14:54:45.872632+0000","last_unstale":"2024-01-16T14:54:46.853602+0000","last_undegraded":"2024-01-16T14:54:46.853602+0000","last_fullsized":"2024-01-16T14:54:46.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261375+0000","last_change":"2024-01-16T14:54:44.852946+0000","last_active":"2024-01-16T14:54:47.261375+0000","last_peered":"2024-01-16T14:54:47.261375+0000","last_clean":"2024-01-16T14:54:47.261375+0000","last_became_active":"2024-01-16T14:54:44.852682+0000","last_became_peered":"2024-01-16T14:54:44.852682+0000","last_unstale":"2024-01-16T14:54:47.261375+0000","last_undegraded":"2024-01-16T14:54:47.261375+0000","last_fullsized":"2024-01-16T14:54:47.261375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261947+0000","last_change":"2024-01-16T14:54:45.877152+0000","last_active":"2024-01-16T14:54:47.261947+0000","last_peered":"2024-01-16T14:54:47.261947+0000","last_clean":"2024-01-16T14:54:47.261947+0000","last_became_active":"2024-01-16T14:54:45.876716+0000","last_became_peered":"2024-01-16T14:54:45.876716+0000","last_unstale":"2024-01-16T14:54:47.261947+0000","last_undegraded":"2024-01-16T14:54:47.261947+0000","last_fullsized":"2024-01-16T14:54:47.261947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261401+0000","last_change":"2024-01-16T14:54:44.857725+0000","last_active":"2024-01-16T14:54:47.261401+0000","last_peered":"2024-01-16T14:54:47.261401+0000","last_clean":"2024-01-16T14:54:47.261401+0000","last_became_active":"2024-01-16T14:54:44.857273+0000","last_became_peered":"2024-01-16T14:54:44.857273+0000","last_unstale":"2024-01-16T14:54:47.261401+0000","last_undegraded":"2024-01-16T14:54:47.261401+0000","last_fullsized":"2024-01-16T14:54:47.261401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853385+0000","last_change":"2024-01-16T14:54:45.868095+0000","last_active":"2024-01-16T14:54:46.853385+0000","last_peered":"2024-01-16T14:54:46.853385+0000","last_clean":"2024-01-16T14:54:46.853385+0000","last_became_active":"2024-01-16T14:54:45.867008+0000","last_became_peered":"2024-01-16T14:54:45.867008+0000","last_unstale":"2024-01-16T14:54:46.853385+0000","last_undegraded":"2024-01-16T14:54:46.853385+0000","last_fullsized":"2024-01-16T14:54:46.853385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855353+0000","last_change":"2024-01-16T14:54:44.853674+0000","last_active":"2024-01-16T14:54:46.855353+0000","last_peered":"2024-01-16T14:54:46.855353+0000","last_clean":"2024-01-16T14:54:46.855353+0000","last_became_active":"2024-01-16T14:54:44.853079+0000","last_became_peered":"2024-01-16T14:54:44.853079+0000","last_unstale":"2024-01-16T14:54:46.855353+0000","last_undegraded":"2024-01-16T14:54:46.855353+0000","last_fullsized":"2024-01-16T14:54:46.855353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:31.747933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853861+0000","last_change":"2024-01-16T14:54:45.875678+0000","last_active":"2024-01-16T14:54:46.853861+0000","last_peered":"2024-01-16T14:54:46.853861+0000","last_clean":"2024-01-16T14:54:46.853861+0000","last_became_active":"2024-01-16T14:54:45.875400+0000","last_became_peered":"2024-01-16T14:54:45.875400+0000","last_unstale":"2024-01-16T14:54:46.853861+0000","last_undegraded":"2024-01-16T14:54:46.853861+0000","last_fullsized":"2024-01-16T14:54:46.853861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262644+0000","last_change":"2024-01-16T14:54:44.858592+0000","last_active":"2024-01-16T14:54:47.262644+0000","last_peered":"2024-01-16T14:54:47.262644+0000","last_clean":"2024-01-16T14:54:47.262644+0000","last_became_active":"2024-01-16T14:54:44.858275+0000","last_became_peered":"2024-01-16T14:54:44.858275+0000","last_unstale":"2024-01-16T14:54:47.262644+0000","last_undegraded":"2024-01-16T14:54:47.262644+0000","last_fullsized":"2024-01-16T14:54:47.262644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262293+0000","last_change":"2024-01-16T14:54:45.877869+0000","last_active":"2024-01-16T14:54:47.262293+0000","last_peered":"2024-01-16T14:54:47.262293+0000","last_clean":"2024-01-16T14:54:47.262293+0000","last_became_active":"2024-01-16T14:54:45.876905+0000","last_became_peered":"2024-01-16T14:54:45.876905+0000","last_unstale":"2024-01-16T14:54:47.262293+0000","last_undegraded":"2024-01-16T14:54:47.262293+0000","last_fullsized":"2024-01-16T14:54:47.262293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:47:51.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261976+0000","last_change":"2024-01-16T14:54:44.855986+0000","last_active":"2024-01-16T14:54:47.261976+0000","last_peered":"2024-01-16T14:54:47.261976+0000","last_clean":"2024-01-16T14:54:47.261976+0000","last_became_active":"2024-01-16T14:54:44.855690+0000","last_became_peered":"2024-01-16T14:54:44.855690+0000","last_unstale":"2024-01-16T14:54:47.261976+0000","last_undegraded":"2024-01-16T14:54:47.261976+0000","last_fullsized":"2024-01-16T14:54:47.261976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855013+0000","last_change":"2024-01-16T14:54:45.878441+0000","last_active":"2024-01-16T14:54:46.855013+0000","last_peered":"2024-01-16T14:54:46.855013+0000","last_clean":"2024-01-16T14:54:46.855013+0000","last_became_active":"2024-01-16T14:54:45.878011+0000","last_became_peered":"2024-01-16T14:54:45.878011+0000","last_unstale":"2024-01-16T14:54:46.855013+0000","last_undegraded":"2024-01-16T14:54:46.855013+0000","last_fullsized":"2024-01-16T14:54:46.855013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:05.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261593+0000","last_change":"2024-01-16T14:54:45.869056+0000","last_active":"2024-01-16T14:54:47.261593+0000","last_peered":"2024-01-16T14:54:47.261593+0000","last_clean":"2024-01-16T14:54:47.261593+0000","last_became_active":"2024-01-16T14:54:45.868748+0000","last_became_peered":"2024-01-16T14:54:45.868748+0000","last_unstale":"2024-01-16T14:54:47.261593+0000","last_undegraded":"2024-01-16T14:54:47.261593+0000","last_fullsized":"2024-01-16T14:54:47.261593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261832+0000","last_change":"2024-01-16T14:54:44.853961+0000","last_active":"2024-01-16T14:54:47.261832+0000","last_peered":"2024-01-16T14:54:47.261832+0000","last_clean":"2024-01-16T14:54:47.261832+0000","last_became_active":"2024-01-16T14:54:44.853610+0000","last_became_peered":"2024-01-16T14:54:44.853610+0000","last_unstale":"2024-01-16T14:54:47.261832+0000","last_undegraded":"2024-01-16T14:54:47.261832+0000","last_fullsized":"2024-01-16T14:54:47.261832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853929+0000","last_change":"2024-01-16T14:54:45.872254+0000","last_active":"2024-01-16T14:54:46.853929+0000","last_peered":"2024-01-16T14:54:46.853929+0000","last_clean":"2024-01-16T14:54:46.853929+0000","last_became_active":"2024-01-16T14:54:45.870597+0000","last_became_peered":"2024-01-16T14:54:45.870597+0000","last_unstale":"2024-01-16T14:54:46.853929+0000","last_undegraded":"2024-01-16T14:54:46.853929+0000","last_fullsized":"2024-01-16T14:54:46.853929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261437+0000","last_change":"2024-01-16T14:54:44.854781+0000","last_active":"2024-01-16T14:54:47.261437+0000","last_peered":"2024-01-16T14:54:47.261437+0000","last_clean":"2024-01-16T14:54:47.261437+0000","last_became_active":"2024-01-16T14:54:44.854461+0000","last_became_peered":"2024-01-16T14:54:44.854461+0000","last_unstale":"2024-01-16T14:54:47.261437+0000","last_undegraded":"2024-01-16T14:54:47.261437+0000","last_fullsized":"2024-01-16T14:54:47.261437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T14:54:45.859524+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:54:45.859524+0000","last_peered":"2024-01-16T14:54:45.859524+0000","last_clean":"2024-01-16T14:54:45.859524+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:54:45.859524+0000","last_undegraded":"2024-01-16T14:54:45.859524+0000","last_fullsized":"2024-01-16T14:54:45.859524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261727+0000","last_change":"2024-01-16T14:54:45.875199+0000","last_active":"2024-01-16T14:54:47.261727+0000","last_peered":"2024-01-16T14:54:47.261727+0000","last_clean":"2024-01-16T14:54:47.261727+0000","last_became_active":"2024-01-16T14:54:45.874049+0000","last_became_peered":"2024-01-16T14:54:45.874049+0000","last_unstale":"2024-01-16T14:54:47.261727+0000","last_undegraded":"2024-01-16T14:54:47.261727+0000","last_fullsized":"2024-01-16T14:54:47.261727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:29.976614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853314+0000","last_change":"2024-01-16T14:54:44.857691+0000","last_active":"2024-01-16T14:54:46.853314+0000","last_peered":"2024-01-16T14:54:46.853314+0000","last_clean":"2024-01-16T14:54:46.853314+0000","last_became_active":"2024-01-16T14:54:44.857407+0000","last_became_peered":"2024-01-16T14:54:44.857407+0000","last_unstale":"2024-01-16T14:54:46.853314+0000","last_undegraded":"2024-01-16T14:54:46.853314+0000","last_fullsized":"2024-01-16T14:54:46.853314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853856+0000","last_change":"2024-01-16T14:54:44.853542+0000","last_active":"2024-01-16T14:54:46.853856+0000","last_peered":"2024-01-16T14:54:46.853856+0000","last_clean":"2024-01-16T14:54:46.853856+0000","last_became_active":"2024-01-16T14:54:44.853273+0000","last_became_peered":"2024-01-16T14:54:44.853273+0000","last_unstale":"2024-01-16T14:54:46.853856+0000","last_undegraded":"2024-01-16T14:54:46.853856+0000","last_fullsized":"2024-01-16T14:54:46.853856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:56.322689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262718+0000","last_change":"2024-01-16T14:54:45.868977+0000","last_active":"2024-01-16T14:54:47.262718+0000","last_peered":"2024-01-16T14:54:47.262718+0000","last_clean":"2024-01-16T14:54:47.262718+0000","last_became_active":"2024-01-16T14:54:45.868675+0000","last_became_peered":"2024-01-16T14:54:45.868675+0000","last_unstale":"2024-01-16T14:54:47.262718+0000","last_undegraded":"2024-01-16T14:54:47.262718+0000","last_fullsized":"2024-01-16T14:54:47.262718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:48.406590+0000","last_change":"2024-01-16T14:54:44.855466+0000","last_active":"2024-01-16T14:54:48.406590+0000","last_peered":"2024-01-16T14:54:48.406590+0000","last_clean":"2024-01-16T14:54:48.406590+0000","last_became_active":"2024-01-16T14:54:44.855144+0000","last_became_peered":"2024-01-16T14:54:44.855144+0000","last_unstale":"2024-01-16T14:54:48.406590+0000","last_undegraded":"2024-01-16T14:54:48.406590+0000","last_fullsized":"2024-01-16T14:54:48.406590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854544+0000","last_change":"2024-01-16T14:54:45.875954+0000","last_active":"2024-01-16T14:54:46.854544+0000","last_peered":"2024-01-16T14:54:46.854544+0000","last_clean":"2024-01-16T14:54:46.854544+0000","last_became_active":"2024-01-16T14:54:45.875635+0000","last_became_peered":"2024-01-16T14:54:45.875635+0000","last_unstale":"2024-01-16T14:54:46.854544+0000","last_undegraded":"2024-01-16T14:54:46.854544+0000","last_fullsized":"2024-01-16T14:54:46.854544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262344+0000","last_change":"2024-01-16T14:54:44.858826+0000","last_active":"2024-01-16T14:54:47.262344+0000","last_peered":"2024-01-16T14:54:47.262344+0000","last_clean":"2024-01-16T14:54:47.262344+0000","last_became_active":"2024-01-16T14:54:44.858528+0000","last_became_peered":"2024-01-16T14:54:44.858528+0000","last_unstale":"2024-01-16T14:54:47.262344+0000","last_undegraded":"2024-01-16T14:54:47.262344+0000","last_fullsized":"2024-01-16T14:54:47.262344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262074+0000","last_change":"2024-01-16T14:54:45.872205+0000","last_active":"2024-01-16T14:54:47.262074+0000","last_peered":"2024-01-16T14:54:47.262074+0000","last_clean":"2024-01-16T14:54:47.262074+0000","last_became_active":"2024-01-16T14:54:45.871893+0000","last_became_peered":"2024-01-16T14:54:45.871893+0000","last_unstale":"2024-01-16T14:54:47.262074+0000","last_undegraded":"2024-01-16T14:54:47.262074+0000","last_fullsized":"2024-01-16T14:54:47.262074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853147+0000","last_change":"2024-01-16T14:54:44.853929+0000","last_active":"2024-01-16T14:54:46.853147+0000","last_peered":"2024-01-16T14:54:46.853147+0000","last_clean":"2024-01-16T14:54:46.853147+0000","last_became_active":"2024-01-16T14:54:44.853655+0000","last_became_peered":"2024-01-16T14:54:44.853655+0000","last_unstale":"2024-01-16T14:54:46.853147+0000","last_undegraded":"2024-01-16T14:54:46.853147+0000","last_fullsized":"2024-01-16T14:54:46.853147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855031+0000","last_change":"2024-01-16T14:54:45.870957+0000","last_active":"2024-01-16T14:54:46.855031+0000","last_peered":"2024-01-16T14:54:46.855031+0000","last_clean":"2024-01-16T14:54:46.855031+0000","last_became_active":"2024-01-16T14:54:45.870672+0000","last_became_peered":"2024-01-16T14:54:45.870672+0000","last_unstale":"2024-01-16T14:54:46.855031+0000","last_undegraded":"2024-01-16T14:54:46.855031+0000","last_fullsized":"2024-01-16T14:54:46.855031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:19.153610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261884+0000","last_change":"2024-01-16T14:54:44.859174+0000","last_active":"2024-01-16T14:54:47.261884+0000","last_peered":"2024-01-16T14:54:47.261884+0000","last_clean":"2024-01-16T14:54:47.261884+0000","last_became_active":"2024-01-16T14:54:44.858845+0000","last_became_peered":"2024-01-16T14:54:44.858845+0000","last_unstale":"2024-01-16T14:54:47.261884+0000","last_undegraded":"2024-01-16T14:54:47.261884+0000","last_fullsized":"2024-01-16T14:54:47.261884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853729+0000","last_change":"2024-01-16T14:54:45.865608+0000","last_active":"2024-01-16T14:54:46.853729+0000","last_peered":"2024-01-16T14:54:46.853729+0000","last_clean":"2024-01-16T14:54:46.853729+0000","last_became_active":"2024-01-16T14:54:45.864856+0000","last_became_peered":"2024-01-16T14:54:45.864856+0000","last_unstale":"2024-01-16T14:54:46.853729+0000","last_undegraded":"2024-01-16T14:54:46.853729+0000","last_fullsized":"2024-01-16T14:54:46.853729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262559+0000","last_change":"2024-01-16T14:54:44.857846+0000","last_active":"2024-01-16T14:54:47.262559+0000","last_peered":"2024-01-16T14:54:47.262559+0000","last_clean":"2024-01-16T14:54:47.262559+0000","last_became_active":"2024-01-16T14:54:44.857244+0000","last_became_peered":"2024-01-16T14:54:44.857244+0000","last_unstale":"2024-01-16T14:54:47.262559+0000","last_undegraded":"2024-01-16T14:54:47.262559+0000","last_fullsized":"2024-01-16T14:54:47.262559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854055+0000","last_change":"2024-01-16T14:54:45.877240+0000","last_active":"2024-01-16T14:54:46.854055+0000","last_peered":"2024-01-16T14:54:46.854055+0000","last_clean":"2024-01-16T14:54:46.854055+0000","last_became_active":"2024-01-16T14:54:45.876921+0000","last_became_peered":"2024-01-16T14:54:45.876921+0000","last_unstale":"2024-01-16T14:54:46.854055+0000","last_undegraded":"2024-01-16T14:54:46.854055+0000","last_fullsized":"2024-01-16T14:54:46.854055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853720+0000","last_change":"2024-01-16T14:54:44.841886+0000","last_active":"2024-01-16T14:54:46.853720+0000","last_peered":"2024-01-16T14:54:46.853720+0000","last_clean":"2024-01-16T14:54:46.853720+0000","last_became_active":"2024-01-16T14:54:44.841590+0000","last_became_peered":"2024-01-16T14:54:44.841590+0000","last_unstale":"2024-01-16T14:54:46.853720+0000","last_undegraded":"2024-01-16T14:54:46.853720+0000","last_fullsized":"2024-01-16T14:54:46.853720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261806+0000","last_change":"2024-01-16T14:54:45.861293+0000","last_active":"2024-01-16T14:54:47.261806+0000","last_peered":"2024-01-16T14:54:47.261806+0000","last_clean":"2024-01-16T14:54:47.261806+0000","last_became_active":"2024-01-16T14:54:45.860966+0000","last_became_peered":"2024-01-16T14:54:45.860966+0000","last_unstale":"2024-01-16T14:54:47.261806+0000","last_undegraded":"2024-01-16T14:54:47.261806+0000","last_fullsized":"2024-01-16T14:54:47.261806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262495+0000","last_change":"2024-01-16T14:54:44.859596+0000","last_active":"2024-01-16T14:54:47.262495+0000","last_peered":"2024-01-16T14:54:47.262495+0000","last_clean":"2024-01-16T14:54:47.262495+0000","last_became_active":"2024-01-16T14:54:44.859150+0000","last_became_peered":"2024-01-16T14:54:44.859150+0000","last_unstale":"2024-01-16T14:54:47.262495+0000","last_undegraded":"2024-01-16T14:54:47.262495+0000","last_fullsized":"2024-01-16T14:54:47.262495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853762+0000","last_change":"2024-01-16T14:54:45.884933+0000","last_active":"2024-01-16T14:54:46.853762+0000","last_peered":"2024-01-16T14:54:46.853762+0000","last_clean":"2024-01-16T14:54:46.853762+0000","last_became_active":"2024-01-16T14:54:45.884688+0000","last_became_peered":"2024-01-16T14:54:45.884688+0000","last_unstale":"2024-01-16T14:54:46.853762+0000","last_undegraded":"2024-01-16T14:54:46.853762+0000","last_fullsized":"2024-01-16T14:54:46.853762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262635+0000","last_change":"2024-01-16T14:54:44.852405+0000","last_active":"2024-01-16T14:54:47.262635+0000","last_peered":"2024-01-16T14:54:47.262635+0000","last_clean":"2024-01-16T14:54:47.262635+0000","last_became_active":"2024-01-16T14:54:44.852011+0000","last_became_peered":"2024-01-16T14:54:44.852011+0000","last_unstale":"2024-01-16T14:54:47.262635+0000","last_undegraded":"2024-01-16T14:54:47.262635+0000","last_fullsized":"2024-01-16T14:54:47.262635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261800+0000","last_change":"2024-01-16T14:54:45.883630+0000","last_active":"2024-01-16T14:54:47.261800+0000","last_peered":"2024-01-16T14:54:47.261800+0000","last_clean":"2024-01-16T14:54:47.261800+0000","last_became_active":"2024-01-16T14:54:45.882901+0000","last_became_peered":"2024-01-16T14:54:45.882901+0000","last_unstale":"2024-01-16T14:54:47.261800+0000","last_undegraded":"2024-01-16T14:54:47.261800+0000","last_fullsized":"2024-01-16T14:54:47.261800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262954+0000","last_change":"2024-01-16T14:54:44.853825+0000","last_active":"2024-01-16T14:54:47.262954+0000","last_peered":"2024-01-16T14:54:47.262954+0000","last_clean":"2024-01-16T14:54:47.262954+0000","last_became_active":"2024-01-16T14:54:44.853509+0000","last_became_peered":"2024-01-16T14:54:44.853509+0000","last_unstale":"2024-01-16T14:54:47.262954+0000","last_undegraded":"2024-01-16T14:54:47.262954+0000","last_fullsized":"2024-01-16T14:54:47.262954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853960+0000","last_change":"2024-01-16T14:54:45.878029+0000","last_active":"2024-01-16T14:54:46.853960+0000","last_peered":"2024-01-16T14:54:46.853960+0000","last_clean":"2024-01-16T14:54:46.853960+0000","last_became_active":"2024-01-16T14:54:45.877266+0000","last_became_peered":"2024-01-16T14:54:45.877266+0000","last_unstale":"2024-01-16T14:54:46.853960+0000","last_undegraded":"2024-01-16T14:54:46.853960+0000","last_fullsized":"2024-01-16T14:54:46.853960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261228+0000","last_change":"2024-01-16T14:54:45.876962+0000","last_active":"2024-01-16T14:54:47.261228+0000","last_peered":"2024-01-16T14:54:47.261228+0000","last_clean":"2024-01-16T14:54:47.261228+0000","last_became_active":"2024-01-16T14:54:45.876358+0000","last_became_peered":"2024-01-16T14:54:45.876358+0000","last_unstale":"2024-01-16T14:54:47.261228+0000","last_undegraded":"2024-01-16T14:54:47.261228+0000","last_fullsized":"2024-01-16T14:54:47.261228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261492+0000","last_change":"2024-01-16T14:54:45.874788+0000","last_active":"2024-01-16T14:54:47.261492+0000","last_peered":"2024-01-16T14:54:47.261492+0000","last_clean":"2024-01-16T14:54:47.261492+0000","last_became_active":"2024-01-16T14:54:45.874453+0000","last_became_peered":"2024-01-16T14:54:45.874453+0000","last_unstale":"2024-01-16T14:54:47.261492+0000","last_undegraded":"2024-01-16T14:54:47.261492+0000","last_fullsized":"2024-01-16T14:54:47.261492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261754+0000","last_change":"2024-01-16T14:54:45.877374+0000","last_active":"2024-01-16T14:54:47.261754+0000","last_peered":"2024-01-16T14:54:47.261754+0000","last_clean":"2024-01-16T14:54:47.261754+0000","last_became_active":"2024-01-16T14:54:45.877118+0000","last_became_peered":"2024-01-16T14:54:45.877118+0000","last_unstale":"2024-01-16T14:54:47.261754+0000","last_undegraded":"2024-01-16T14:54:47.261754+0000","last_fullsized":"2024-01-16T14:54:47.261754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262220+0000","last_change":"2024-01-16T14:54:45.875476+0000","last_active":"2024-01-16T14:54:47.262220+0000","last_peered":"2024-01-16T14:54:47.262220+0000","last_clean":"2024-01-16T14:54:47.262220+0000","last_became_active":"2024-01-16T14:54:45.874303+0000","last_became_peered":"2024-01-16T14:54:45.874303+0000","last_unstale":"2024-01-16T14:54:47.262220+0000","last_undegraded":"2024-01-16T14:54:47.262220+0000","last_fullsized":"2024-01-16T14:54:47.262220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854023+0000","last_change":"2024-01-16T14:54:45.866307+0000","last_active":"2024-01-16T14:54:46.854023+0000","last_peered":"2024-01-16T14:54:46.854023+0000","last_clean":"2024-01-16T14:54:46.854023+0000","last_became_active":"2024-01-16T14:54:45.865419+0000","last_became_peered":"2024-01-16T14:54:45.865419+0000","last_unstale":"2024-01-16T14:54:46.854023+0000","last_undegraded":"2024-01-16T14:54:46.854023+0000","last_fullsized":"2024-01-16T14:54:46.854023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262723+0000","last_change":"2024-01-16T14:54:45.866420+0000","last_active":"2024-01-16T14:54:47.262723+0000","last_peered":"2024-01-16T14:54:47.262723+0000","last_clean":"2024-01-16T14:54:47.262723+0000","last_became_active":"2024-01-16T14:54:45.866022+0000","last_became_peered":"2024-01-16T14:54:45.866022+0000","last_unstale":"2024-01-16T14:54:47.262723+0000","last_undegraded":"2024-01-16T14:54:47.262723+0000","last_fullsized":"2024-01-16T14:54:47.262723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262181+0000","last_change":"2024-01-16T14:54:45.875523+0000","last_active":"2024-01-16T14:54:47.262181+0000","last_peered":"2024-01-16T14:54:47.262181+0000","last_clean":"2024-01-16T14:54:47.262181+0000","last_became_active":"2024-01-16T14:54:45.875252+0000","last_became_peered":"2024-01-16T14:54:45.875252+0000","last_unstale":"2024-01-16T14:54:47.262181+0000","last_undegraded":"2024-01-16T14:54:47.262181+0000","last_fullsized":"2024-01-16T14:54:47.262181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854401+0000","last_change":"2024-01-16T14:54:45.865807+0000","last_active":"2024-01-16T14:54:46.854401+0000","last_peered":"2024-01-16T14:54:46.854401+0000","last_clean":"2024-01-16T14:54:46.854401+0000","last_became_active":"2024-01-16T14:54:45.865257+0000","last_became_peered":"2024-01-16T14:54:45.865257+0000","last_unstale":"2024-01-16T14:54:46.854401+0000","last_undegraded":"2024-01-16T14:54:46.854401+0000","last_fullsized":"2024-01-16T14:54:46.854401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261310+0000","last_change":"2024-01-16T14:54:45.883032+0000","last_active":"2024-01-16T14:54:47.261310+0000","last_peered":"2024-01-16T14:54:47.261310+0000","last_clean":"2024-01-16T14:54:47.261310+0000","last_became_active":"2024-01-16T14:54:45.882768+0000","last_became_peered":"2024-01-16T14:54:45.882768+0000","last_unstale":"2024-01-16T14:54:47.261310+0000","last_undegraded":"2024-01-16T14:54:47.261310+0000","last_fullsized":"2024-01-16T14:54:47.261310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853768+0000","last_change":"2024-01-16T14:54:45.866167+0000","last_active":"2024-01-16T14:54:46.853768+0000","last_peered":"2024-01-16T14:54:46.853768+0000","last_clean":"2024-01-16T14:54:46.853768+0000","last_became_active":"2024-01-16T14:54:45.865393+0000","last_became_peered":"2024-01-16T14:54:45.865393+0000","last_unstale":"2024-01-16T14:54:46.853768+0000","last_undegraded":"2024-01-16T14:54:46.853768+0000","last_fullsized":"2024-01-16T14:54:46.853768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261572+0000","last_change":"2024-01-16T14:54:45.882612+0000","last_active":"2024-01-16T14:54:47.261572+0000","last_peered":"2024-01-16T14:54:47.261572+0000","last_clean":"2024-01-16T14:54:47.261572+0000","last_became_active":"2024-01-16T14:54:45.882301+0000","last_became_peered":"2024-01-16T14:54:45.882301+0000","last_unstale":"2024-01-16T14:54:47.261572+0000","last_undegraded":"2024-01-16T14:54:47.261572+0000","last_fullsized":"2024-01-16T14:54:47.261572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262090+0000","last_change":"2024-01-16T14:54:45.876292+0000","last_active":"2024-01-16T14:54:47.262090+0000","last_peered":"2024-01-16T14:54:47.262090+0000","last_clean":"2024-01-16T14:54:47.262090+0000","last_became_active":"2024-01-16T14:54:45.875995+0000","last_became_peered":"2024-01-16T14:54:45.875995+0000","last_unstale":"2024-01-16T14:54:47.262090+0000","last_undegraded":"2024-01-16T14:54:47.262090+0000","last_fullsized":"2024-01-16T14:54:47.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261512+0000","last_change":"2024-01-16T14:54:45.866707+0000","last_active":"2024-01-16T14:54:47.261512+0000","last_peered":"2024-01-16T14:54:47.261512+0000","last_clean":"2024-01-16T14:54:47.261512+0000","last_became_active":"2024-01-16T14:54:45.866333+0000","last_became_peered":"2024-01-16T14:54:45.866333+0000","last_unstale":"2024-01-16T14:54:47.261512+0000","last_undegraded":"2024-01-16T14:54:47.261512+0000","last_fullsized":"2024-01-16T14:54:47.261512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:54:55.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:54:55.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.343+0000 7f91fb31c700 1 Processor -- start 2024-01-16T14:54:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.344+0000 7f91fb31c700 1 -- start start 2024-01-16T14:54:55.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91fb31c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f91f40ae970 0x7f91f4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91fb31c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40af770 con 0x7f91f415aa10 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91fb31c700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40afa50 con 0x7f91f415a630 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91fb31c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f40ad540 con 0x7f91f40ae970 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f88b7700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91f415aa10 0x7f91f40b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58572/0 (socket says 172.21.15.26:58572) 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f88b7700 1 -- 172.21.15.26:0/3082032535 learned_addr learned my addr 172.21.15.26:0/3082032535 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f98b9700 1 --2- 172.21.15.26:0/3082032535 >> v2:172.21.15.93:3300/0 conn(0x7f91f40ae970 0x7f91f4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91ebfff700 1 -- 172.21.15.26:0/3082032535 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346406295 0 0) 0x7f91f40af770 con 0x7f91f415aa10 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91ebfff700 1 -- 172.21.15.26:0/3082032535 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e00046c0 con 0x7f91f415aa10 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f98b9700 1 -- 172.21.15.26:0/3082032535 >> v1:172.21.15.93:6789/0 conn(0x7f91f415a630 legacy=0x7f91f40b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f98b9700 1 -- 172.21.15.26:0/3082032535 >> v1:172.21.15.26:6789/0 conn(0x7f91f415aa10 legacy=0x7f91f40b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f98b9700 1 -- 172.21.15.26:0/3082032535 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40ad160 con 0x7f91f40ae970 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.345+0000 7f91f98b9700 1 --2- 172.21.15.26:0/3082032535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f40ae970 0x7f91f4072710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009d90 tx=0x7f91ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d3d9bf39ad068b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91ebfff700 1 -- 172.21.15.26:0/3082032535 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00e070 con 0x7f91f40ae970 2024-01-16T14:54:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91ebfff700 1 -- 172.21.15.26:0/3082032535 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec00bb20 con 0x7f91f40ae970 2024-01-16T14:54:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91ebfff700 1 -- 172.21.15.26:0/3082032535 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec012430 con 0x7f91f40ae970 2024-01-16T14:54:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 -- 172.21.15.26:0/3082032535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f40ae970 msgr2=0x7f91f4072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 --2- 172.21.15.26:0/3082032535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f40ae970 0x7f91f4072710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009d90 tx=0x7f91ec00b330 comp rx=0 tx=0).stop 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 -- 172.21.15.26:0/3082032535 shutdown_connections 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 --2- 172.21.15.26:0/3082032535 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f40ae970 0x7f91f4072710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 -- 172.21.15.26:0/3082032535 >> 172.21.15.26:0/3082032535 conn(0x7f91f40b3190 msgr2=0x7f91f40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 -- 172.21.15.26:0/3082032535 shutdown_connections 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.346+0000 7f91fb31c700 1 -- 172.21.15.26:0/3082032535 wait complete. 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 Processor -- start 2024-01-16T14:54:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 -- start start 2024-01-16T14:54:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91f415a630 0x7f91f41580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f415aa10 0x7f91f40cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41e3ff0 con 0x7f91f40ae970 2024-01-16T14:54:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41e42c0 con 0x7f91f415aa10 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91fb31c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f41e45a0 con 0x7f91f415a630 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91f90b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.348+0000 7f91f90b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55338/0 (socket says 172.21.15.26:55338) 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 -- 172.21.15.26:0/719434447 learned_addr learned my addr 172.21.15.26:0/719434447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f98b9700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f415aa10 0x7f91f40cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f88b7700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91f415a630 0x7f91f41580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 -- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91f415a630 msgr2=0x7f91f41580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91f415a630 0x7f91f41580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 -- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f415aa10 msgr2=0x7f91f40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f415aa10 0x7f91f40cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 -- 172.21.15.26:0/719434447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec009a70 con 0x7f91f40ae970 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91f90b8700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c4e0 tx=0x7f91f000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f00052d0 con 0x7f91f40ae970 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f0005430 con 0x7f91f40ae970 2024-01-16T14:54:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f41e4880 con 0x7f91f40ae970 2024-01-16T14:54:55.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.349+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f00038c0 con 0x7f91f40ae970 2024-01-16T14:54:55.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.350+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f41e4e40 con 0x7f91f40ae970 2024-01-16T14:54:55.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.350+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f4075480 con 0x7f91f40ae970 2024-01-16T14:54:55.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.351+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f91f0003a20 con 0x7f91f40ae970 2024-01-16T14:54:55.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.352+0000 7f91e9ffb700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 0x7f91d806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.352+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91f00892c0 con 0x7f91f40ae970 2024-01-16T14:54:55.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.352+0000 7f91f88b7700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 0x7f91d806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.352+0000 7f91f88b7700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 0x7f91d806d810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007fd0 tx=0x7f91e4007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.356+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91f0054210 con 0x7f91f40ae970 2024-01-16T14:54:55.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.563+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f4068af0 con 0x7f91d806b350 2024-01-16T14:54:55.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.571+0000 7f91e9ffb700 1 -- 172.21.15.26:0/719434447 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264478 (secure 0 0 0) 0x7f91f4068af0 con 0x7f91d806b350 2024-01-16T14:54:55.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 msgr2=0x7f91d806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 0x7f91d806d810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007fd0 tx=0x7f91e4007dc0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 msgr2=0x7f91f41e3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c4e0 tx=0x7f91f000c7f0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 shutdown_connections 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91f415aa10 0x7f91f40cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91d806b350 0x7f91d806d810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91f40ae970 0x7f91f41e3ab0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 --2- 172.21.15.26:0/719434447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91f415a630 0x7f91f41580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.574+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 >> 172.21.15.26:0/719434447 conn(0x7f91f40b3190 msgr2=0x7f91f4158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:55.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.575+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 shutdown_connections 2024-01-16T14:54:55.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.575+0000 7f91fb31c700 1 -- 172.21.15.26:0/719434447 wait complete. 2024-01-16T14:54:55.563 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:54:55.571 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-16T14:54:55.352370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2973696,"data_stored":1787728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.501465"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854111+0000","last_change":"2024-01-16T14:54:45.877716+0000","last_active":"2024-01-16T14:54:46.854111+0000","last_peered":"2024-01-16T14:54:46.854111+0000","last_clean":"2024-01-16T14:54:46.854111+0000","last_became_active":"2024-01-16T14:54:45.876462+0000","last_became_peered":"2024-01-16T14:54:45.876462+0000","last_unstale":"2024-01-16T14:54:46.854111+0000","last_undegraded":"2024-01-16T14:54:46.854111+0000","last_fullsized":"2024-01-16T14:54:46.854111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262923+0000","last_change":"2024-01-16T14:54:45.880623+0000","last_active":"2024-01-16T14:54:47.262923+0000","last_peered":"2024-01-16T14:54:47.262923+0000","last_clean":"2024-01-16T14:54:47.262923+0000","last_became_active":"2024-01-16T14:54:45.878584+0000","last_became_peered":"2024-01-16T14:54:45.878584+0000","last_unstale":"2024-01-16T14:54:47.262923+0000","last_undegraded":"2024-01-16T14:54:47.262923+0000","last_fullsized":"2024-01-16T14:54:47.262923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261939+0000","last_change":"2024-01-16T14:54:45.875065+0000","last_active":"2024-01-16T14:54:47.261939+0000","last_peered":"2024-01-16T14:54:47.261939+0000","last_clean":"2024-01-16T14:54:47.261939+0000","last_became_active":"2024-01-16T14:54:45.873809+0000","last_became_peered":"2024-01-16T14:54:45.873809+0000","last_unstale":"2024-01-16T14:54:47.261939+0000","last_undegraded":"2024-01-16T14:54:47.261939+0000","last_fullsized":"2024-01-16T14:54:47.261939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261991+0000","last_change":"2024-01-16T14:54:45.877233+0000","last_active":"2024-01-16T14:54:47.261991+0000","last_peered":"2024-01-16T14:54:47.261991+0000","last_clean":"2024-01-16T14:54:47.261991+0000","last_became_active":"2024-01-16T14:54:45.876940+0000","last_became_peered":"2024-01-16T14:54:45.876940+0000","last_unstale":"2024-01-16T14:54:47.261991+0000","last_undegraded":"2024-01-16T14:54:47.261991+0000","last_fullsized":"2024-01-16T14:54:47.261991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853429+0000","last_change":"2024-01-16T14:54:45.874709+0000","last_active":"2024-01-16T14:54:46.853429+0000","last_peered":"2024-01-16T14:54:46.853429+0000","last_clean":"2024-01-16T14:54:46.853429+0000","last_became_active":"2024-01-16T14:54:45.872900+0000","last_became_peered":"2024-01-16T14:54:45.872900+0000","last_unstale":"2024-01-16T14:54:46.853429+0000","last_undegraded":"2024-01-16T14:54:46.853429+0000","last_fullsized":"2024-01-16T14:54:46.853429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261215+0000","last_change":"2024-01-16T14:54:45.882475+0000","last_active":"2024-01-16T14:54:47.261215+0000","last_peered":"2024-01-16T14:54:47.261215+0000","last_clean":"2024-01-16T14:54:47.261215+0000","last_became_active":"2024-01-16T14:54:45.882150+0000","last_became_peered":"2024-01-16T14:54:45.882150+0000","last_unstale":"2024-01-16T14:54:47.261215+0000","last_undegraded":"2024-01-16T14:54:47.261215+0000","last_fullsized":"2024-01-16T14:54:47.261215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853589+0000","last_change":"2024-01-16T14:54:45.875750+0000","last_active":"2024-01-16T14:54:46.853589+0000","last_peered":"2024-01-16T14:54:46.853589+0000","last_clean":"2024-01-16T14:54:46.853589+0000","last_became_active":"2024-01-16T14:54:45.875474+0000","last_became_peered":"2024-01-16T14:54:45.875474+0000","last_unstale":"2024-01-16T14:54:46.853589+0000","last_undegraded":"2024-01-16T14:54:46.853589+0000","last_fullsized":"2024-01-16T14:54:46.853589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853586+0000","last_change":"2024-01-16T14:54:45.882976+0000","last_active":"2024-01-16T14:54:46.853586+0000","last_peered":"2024-01-16T14:54:46.853586+0000","last_clean":"2024-01-16T14:54:46.853586+0000","last_became_active":"2024-01-16T14:54:45.882628+0000","last_became_peered":"2024-01-16T14:54:45.882628+0000","last_unstale":"2024-01-16T14:54:46.853586+0000","last_undegraded":"2024-01-16T14:54:46.853586+0000","last_fullsized":"2024-01-16T14:54:46.853586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853730+0000","last_change":"2024-01-16T14:54:45.885967+0000","last_active":"2024-01-16T14:54:46.853730+0000","last_peered":"2024-01-16T14:54:46.853730+0000","last_clean":"2024-01-16T14:54:46.853730+0000","last_became_active":"2024-01-16T14:54:45.885700+0000","last_became_peered":"2024-01-16T14:54:45.885700+0000","last_unstale":"2024-01-16T14:54:46.853730+0000","last_undegraded":"2024-01-16T14:54:46.853730+0000","last_fullsized":"2024-01-16T14:54:46.853730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262320+0000","last_change":"2024-01-16T14:54:45.878748+0000","last_active":"2024-01-16T14:54:47.262320+0000","last_peered":"2024-01-16T14:54:47.262320+0000","last_clean":"2024-01-16T14:54:47.262320+0000","last_became_active":"2024-01-16T14:54:45.878446+0000","last_became_peered":"2024-01-16T14:54:45.878446+0000","last_unstale":"2024-01-16T14:54:47.262320+0000","last_undegraded":"2024-01-16T14:54:47.262320+0000","last_fullsized":"2024-01-16T14:54:47.262320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261357+0000","last_change":"2024-01-16T14:54:45.879232+0000","last_active":"2024-01-16T14:54:47.261357+0000","last_peered":"2024-01-16T14:54:47.261357+0000","last_clean":"2024-01-16T14:54:47.261357+0000","last_became_active":"2024-01-16T14:54:45.878875+0000","last_became_peered":"2024-01-16T14:54:45.878875+0000","last_unstale":"2024-01-16T14:54:47.261357+0000","last_undegraded":"2024-01-16T14:54:47.261357+0000","last_fullsized":"2024-01-16T14:54:47.261357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855341+0000","last_change":"2024-01-16T14:54:45.862373+0000","last_active":"2024-01-16T14:54:46.855341+0000","last_peered":"2024-01-16T14:54:46.855341+0000","last_clean":"2024-01-16T14:54:46.855341+0000","last_became_active":"2024-01-16T14:54:45.862110+0000","last_became_peered":"2024-01-16T14:54:45.862110+0000","last_unstale":"2024-01-16T14:54:46.855341+0000","last_undegraded":"2024-01-16T14:54:46.855341+0000","last_fullsized":"2024-01-16T14:54:46.855341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854063+0000","last_change":"2024-01-16T14:54:45.867064+0000","last_active":"2024-01-16T14:54:46.854063+0000","last_peered":"2024-01-16T14:54:46.854063+0000","last_clean":"2024-01-16T14:54:46.854063+0000","last_became_active":"2024-01-16T14:54:45.866257+0000","last_became_peered":"2024-01-16T14:54:45.866257+0000","last_unstale":"2024-01-16T14:54:46.854063+0000","last_undegraded":"2024-01-16T14:54:46.854063+0000","last_fullsized":"2024-01-16T14:54:46.854063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261852+0000","last_change":"2024-01-16T14:54:45.875851+0000","last_active":"2024-01-16T14:54:47.261852+0000","last_peered":"2024-01-16T14:54:47.261852+0000","last_clean":"2024-01-16T14:54:47.261852+0000","last_became_active":"2024-01-16T14:54:45.875202+0000","last_became_peered":"2024-01-16T14:54:45.875202+0000","last_unstale":"2024-01-16T14:54:47.261852+0000","last_undegraded":"2024-01-16T14:54:47.261852+0000","last_fullsized":"2024-01-16T14:54:47.261852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:35.053270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261833+0000","last_change":"2024-01-16T14:54:45.858130+0000","last_active":"2024-01-16T14:54:47.261833+0000","last_peered":"2024-01-16T14:54:47.261833+0000","last_clean":"2024-01-16T14:54:47.261833+0000","last_became_active":"2024-01-16T14:54:45.856812+0000","last_became_peered":"2024-01-16T14:54:45.856812+0000","last_unstale":"2024-01-16T14:54:47.261833+0000","last_undegraded":"2024-01-16T14:54:47.261833+0000","last_fullsized":"2024-01-16T14:54:47.261833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261530+0000","last_change":"2024-01-16T14:54:45.883516+0000","last_active":"2024-01-16T14:54:47.261530+0000","last_peered":"2024-01-16T14:54:47.261530+0000","last_clean":"2024-01-16T14:54:47.261530+0000","last_became_active":"2024-01-16T14:54:45.883259+0000","last_became_peered":"2024-01-16T14:54:45.883259+0000","last_unstale":"2024-01-16T14:54:47.261530+0000","last_undegraded":"2024-01-16T14:54:47.261530+0000","last_fullsized":"2024-01-16T14:54:47.261530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262193+0000","last_change":"2024-01-16T14:54:45.883796+0000","last_active":"2024-01-16T14:54:47.262193+0000","last_peered":"2024-01-16T14:54:47.262193+0000","last_clean":"2024-01-16T14:54:47.262193+0000","last_became_active":"2024-01-16T14:54:45.883558+0000","last_became_peered":"2024-01-16T14:54:45.883558+0000","last_unstale":"2024-01-16T14:54:47.262193+0000","last_undegraded":"2024-01-16T14:54:47.262193+0000","last_fullsized":"2024-01-16T14:54:47.262193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:24:14.693493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262894+0000","last_change":"2024-01-16T14:54:45.867763+0000","last_active":"2024-01-16T14:54:47.262894+0000","last_peered":"2024-01-16T14:54:47.262894+0000","last_clean":"2024-01-16T14:54:47.262894+0000","last_became_active":"2024-01-16T14:54:45.867441+0000","last_became_peered":"2024-01-16T14:54:45.867441+0000","last_unstale":"2024-01-16T14:54:47.262894+0000","last_undegraded":"2024-01-16T14:54:47.262894+0000","last_fullsized":"2024-01-16T14:54:47.262894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853971+0000","last_change":"2024-01-16T14:54:45.873151+0000","last_active":"2024-01-16T14:54:46.853971+0000","last_peered":"2024-01-16T14:54:46.853971+0000","last_clean":"2024-01-16T14:54:46.853971+0000","last_became_active":"2024-01-16T14:54:45.872841+0000","last_became_peered":"2024-01-16T14:54:45.872841+0000","last_unstale":"2024-01-16T14:54:46.853971+0000","last_undegraded":"2024-01-16T14:54:46.853971+0000","last_fullsized":"2024-01-16T14:54:46.853971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262224+0000","last_change":"2024-01-16T14:54:44.860003+0000","last_active":"2024-01-16T14:54:47.262224+0000","last_peered":"2024-01-16T14:54:47.262224+0000","last_clean":"2024-01-16T14:54:47.262224+0000","last_became_active":"2024-01-16T14:54:44.859774+0000","last_became_peered":"2024-01-16T14:54:44.859774+0000","last_unstale":"2024-01-16T14:54:47.262224+0000","last_undegraded":"2024-01-16T14:54:47.262224+0000","last_fullsized":"2024-01-16T14:54:47.262224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854240+0000","last_change":"2024-01-16T14:54:45.867265+0000","last_active":"2024-01-16T14:54:46.854240+0000","last_peered":"2024-01-16T14:54:46.854240+0000","last_clean":"2024-01-16T14:54:46.854240+0000","last_became_active":"2024-01-16T14:54:45.865756+0000","last_became_peered":"2024-01-16T14:54:45.865756+0000","last_unstale":"2024-01-16T14:54:46.854240+0000","last_undegraded":"2024-01-16T14:54:46.854240+0000","last_fullsized":"2024-01-16T14:54:46.854240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:10:44.135270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854088+0000","last_change":"2024-01-16T14:54:44.853662+0000","last_active":"2024-01-16T14:54:46.854088+0000","last_peered":"2024-01-16T14:54:46.854088+0000","last_clean":"2024-01-16T14:54:46.854088+0000","last_became_active":"2024-01-16T14:54:44.853379+0000","last_became_peered":"2024-01-16T14:54:44.853379+0000","last_unstale":"2024-01-16T14:54:46.854088+0000","last_undegraded":"2024-01-16T14:54:46.854088+0000","last_fullsized":"2024-01-16T14:54:46.854088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262160+0000","last_change":"2024-01-16T14:54:45.880017+0000","last_active":"2024-01-16T14:54:47.262160+0000","last_peered":"2024-01-16T14:54:47.262160+0000","last_clean":"2024-01-16T14:54:47.262160+0000","last_became_active":"2024-01-16T14:54:45.879369+0000","last_became_peered":"2024-01-16T14:54:45.879369+0000","last_unstale":"2024-01-16T14:54:47.262160+0000","last_undegraded":"2024-01-16T14:54:47.262160+0000","last_fullsized":"2024-01-16T14:54:47.262160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853963+0000","last_change":"2024-01-16T14:54:44.852663+0000","last_active":"2024-01-16T14:54:46.853963+0000","last_peered":"2024-01-16T14:54:46.853963+0000","last_clean":"2024-01-16T14:54:46.853963+0000","last_became_active":"2024-01-16T14:54:44.852346+0000","last_became_peered":"2024-01-16T14:54:44.852346+0000","last_unstale":"2024-01-16T14:54:46.853963+0000","last_undegraded":"2024-01-16T14:54:46.853963+0000","last_fullsized":"2024-01-16T14:54:46.853963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:54.571295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261811+0000","last_change":"2024-01-16T14:54:45.886559+0000","last_active":"2024-01-16T14:54:47.261811+0000","last_peered":"2024-01-16T14:54:47.261811+0000","last_clean":"2024-01-16T14:54:47.261811+0000","last_became_active":"2024-01-16T14:54:45.886343+0000","last_became_peered":"2024-01-16T14:54:45.886343+0000","last_unstale":"2024-01-16T14:54:47.261811+0000","last_undegraded":"2024-01-16T14:54:47.261811+0000","last_fullsized":"2024-01-16T14:54:47.261811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853331+0000","last_change":"2024-01-16T14:54:44.852311+0000","last_active":"2024-01-16T14:54:46.853331+0000","last_peered":"2024-01-16T14:54:46.853331+0000","last_clean":"2024-01-16T14:54:46.853331+0000","last_became_active":"2024-01-16T14:54:44.852018+0000","last_became_peered":"2024-01-16T14:54:44.852018+0000","last_unstale":"2024-01-16T14:54:46.853331+0000","last_undegraded":"2024-01-16T14:54:46.853331+0000","last_fullsized":"2024-01-16T14:54:46.853331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262110+0000","last_change":"2024-01-16T14:54:45.883384+0000","last_active":"2024-01-16T14:54:47.262110+0000","last_peered":"2024-01-16T14:54:47.262110+0000","last_clean":"2024-01-16T14:54:47.262110+0000","last_became_active":"2024-01-16T14:54:45.883094+0000","last_became_peered":"2024-01-16T14:54:45.883094+0000","last_unstale":"2024-01-16T14:54:47.262110+0000","last_undegraded":"2024-01-16T14:54:47.262110+0000","last_fullsized":"2024-01-16T14:54:47.262110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:03:35.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853863+0000","last_change":"2024-01-16T14:54:44.846655+0000","last_active":"2024-01-16T14:54:46.853863+0000","last_peered":"2024-01-16T14:54:46.853863+0000","last_clean":"2024-01-16T14:54:46.853863+0000","last_became_active":"2024-01-16T14:54:44.846359+0000","last_became_peered":"2024-01-16T14:54:44.846359+0000","last_unstale":"2024-01-16T14:54:46.853863+0000","last_undegraded":"2024-01-16T14:54:46.853863+0000","last_fullsized":"2024-01-16T14:54:46.853863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:28.971339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854823+0000","last_change":"2024-01-16T14:54:45.857956+0000","last_active":"2024-01-16T14:54:46.854823+0000","last_peered":"2024-01-16T14:54:46.854823+0000","last_clean":"2024-01-16T14:54:46.854823+0000","last_became_active":"2024-01-16T14:54:45.857645+0000","last_became_peered":"2024-01-16T14:54:45.857645+0000","last_unstale":"2024-01-16T14:54:46.854823+0000","last_undegraded":"2024-01-16T14:54:46.854823+0000","last_fullsized":"2024-01-16T14:54:46.854823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:39.613003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261584+0000","last_change":"2024-01-16T14:54:44.851910+0000","last_active":"2024-01-16T14:54:47.261584+0000","last_peered":"2024-01-16T14:54:47.261584+0000","last_clean":"2024-01-16T14:54:47.261584+0000","last_became_active":"2024-01-16T14:54:44.851631+0000","last_became_peered":"2024-01-16T14:54:44.851631+0000","last_unstale":"2024-01-16T14:54:47.261584+0000","last_undegraded":"2024-01-16T14:54:47.261584+0000","last_fullsized":"2024-01-16T14:54:47.261584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:17.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262365+0000","last_change":"2024-01-16T14:54:45.875708+0000","last_active":"2024-01-16T14:54:47.262365+0000","last_peered":"2024-01-16T14:54:47.262365+0000","last_clean":"2024-01-16T14:54:47.262365+0000","last_became_active":"2024-01-16T14:54:45.875328+0000","last_became_peered":"2024-01-16T14:54:45.875328+0000","last_unstale":"2024-01-16T14:54:47.262365+0000","last_undegraded":"2024-01-16T14:54:47.262365+0000","last_fullsized":"2024-01-16T14:54:47.262365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853648+0000","last_change":"2024-01-16T14:54:44.855286+0000","last_active":"2024-01-16T14:54:46.853648+0000","last_peered":"2024-01-16T14:54:46.853648+0000","last_clean":"2024-01-16T14:54:46.853648+0000","last_became_active":"2024-01-16T14:54:44.854465+0000","last_became_peered":"2024-01-16T14:54:44.854465+0000","last_unstale":"2024-01-16T14:54:46.853648+0000","last_undegraded":"2024-01-16T14:54:46.853648+0000","last_fullsized":"2024-01-16T14:54:46.853648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853759+0000","last_change":"2024-01-16T14:54:45.868676+0000","last_active":"2024-01-16T14:54:46.853759+0000","last_peered":"2024-01-16T14:54:46.853759+0000","last_clean":"2024-01-16T14:54:46.853759+0000","last_became_active":"2024-01-16T14:54:45.868354+0000","last_became_peered":"2024-01-16T14:54:45.868354+0000","last_unstale":"2024-01-16T14:54:46.853759+0000","last_undegraded":"2024-01-16T14:54:46.853759+0000","last_fullsized":"2024-01-16T14:54:46.853759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261155+0000","last_change":"2024-01-16T14:54:44.855323+0000","last_active":"2024-01-16T14:54:47.261155+0000","last_peered":"2024-01-16T14:54:47.261155+0000","last_clean":"2024-01-16T14:54:47.261155+0000","last_became_active":"2024-01-16T14:54:44.855035+0000","last_became_peered":"2024-01-16T14:54:44.855035+0000","last_unstale":"2024-01-16T14:54:47.261155+0000","last_undegraded":"2024-01-16T14:54:47.261155+0000","last_fullsized":"2024-01-16T14:54:47.261155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:33.068253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262370+0000","last_change":"2024-01-16T14:54:45.864396+0000","last_active":"2024-01-16T14:54:47.262370+0000","last_peered":"2024-01-16T14:54:47.262370+0000","last_clean":"2024-01-16T14:54:47.262370+0000","last_became_active":"2024-01-16T14:54:45.864052+0000","last_became_peered":"2024-01-16T14:54:45.864052+0000","last_unstale":"2024-01-16T14:54:47.262370+0000","last_undegraded":"2024-01-16T14:54:47.262370+0000","last_fullsized":"2024-01-16T14:54:47.262370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262332+0000","last_change":"2024-01-16T14:54:44.857192+0000","last_active":"2024-01-16T14:54:47.262332+0000","last_peered":"2024-01-16T14:54:47.262332+0000","last_clean":"2024-01-16T14:54:47.262332+0000","last_became_active":"2024-01-16T14:54:44.856880+0000","last_became_peered":"2024-01-16T14:54:44.856880+0000","last_unstale":"2024-01-16T14:54:47.262332+0000","last_undegraded":"2024-01-16T14:54:47.262332+0000","last_fullsized":"2024-01-16T14:54:47.262332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:26.907003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261868+0000","last_change":"2024-01-16T14:54:45.884208+0000","last_active":"2024-01-16T14:54:47.261868+0000","last_peered":"2024-01-16T14:54:47.261868+0000","last_clean":"2024-01-16T14:54:47.261868+0000","last_became_active":"2024-01-16T14:54:45.882540+0000","last_became_peered":"2024-01-16T14:54:45.882540+0000","last_unstale":"2024-01-16T14:54:47.261868+0000","last_undegraded":"2024-01-16T14:54:47.261868+0000","last_fullsized":"2024-01-16T14:54:47.261868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261721+0000","last_change":"2024-01-16T14:54:44.854856+0000","last_active":"2024-01-16T14:54:47.261721+0000","last_peered":"2024-01-16T14:54:47.261721+0000","last_clean":"2024-01-16T14:54:47.261721+0000","last_became_active":"2024-01-16T14:54:44.854543+0000","last_became_peered":"2024-01-16T14:54:44.854543+0000","last_unstale":"2024-01-16T14:54:47.261721+0000","last_undegraded":"2024-01-16T14:54:47.261721+0000","last_fullsized":"2024-01-16T14:54:47.261721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262312+0000","last_change":"2024-01-16T14:54:45.880240+0000","last_active":"2024-01-16T14:54:47.262312+0000","last_peered":"2024-01-16T14:54:47.262312+0000","last_clean":"2024-01-16T14:54:47.262312+0000","last_became_active":"2024-01-16T14:54:45.879764+0000","last_became_peered":"2024-01-16T14:54:45.879764+0000","last_unstale":"2024-01-16T14:54:47.262312+0000","last_undegraded":"2024-01-16T14:54:47.262312+0000","last_fullsized":"2024-01-16T14:54:47.262312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262763+0000","last_change":"2024-01-16T14:54:44.855449+0000","last_active":"2024-01-16T14:54:47.262763+0000","last_peered":"2024-01-16T14:54:47.262763+0000","last_clean":"2024-01-16T14:54:47.262763+0000","last_became_active":"2024-01-16T14:54:44.855037+0000","last_became_peered":"2024-01-16T14:54:44.855037+0000","last_unstale":"2024-01-16T14:54:47.262763+0000","last_undegraded":"2024-01-16T14:54:47.262763+0000","last_fullsized":"2024-01-16T14:54:47.262763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855392+0000","last_change":"2024-01-16T14:54:45.870932+0000","last_active":"2024-01-16T14:54:46.855392+0000","last_peered":"2024-01-16T14:54:46.855392+0000","last_clean":"2024-01-16T14:54:46.855392+0000","last_became_active":"2024-01-16T14:54:45.870016+0000","last_became_peered":"2024-01-16T14:54:45.870016+0000","last_unstale":"2024-01-16T14:54:46.855392+0000","last_undegraded":"2024-01-16T14:54:46.855392+0000","last_fullsized":"2024-01-16T14:54:46.855392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:57.092059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261353+0000","last_change":"2024-01-16T14:54:44.854762+0000","last_active":"2024-01-16T14:54:47.261353+0000","last_peered":"2024-01-16T14:54:47.261353+0000","last_clean":"2024-01-16T14:54:47.261353+0000","last_became_active":"2024-01-16T14:54:44.854507+0000","last_became_peered":"2024-01-16T14:54:44.854507+0000","last_unstale":"2024-01-16T14:54:47.261353+0000","last_undegraded":"2024-01-16T14:54:47.261353+0000","last_fullsized":"2024-01-16T14:54:47.261353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853580+0000","last_change":"2024-01-16T14:54:45.879814+0000","last_active":"2024-01-16T14:54:46.853580+0000","last_peered":"2024-01-16T14:54:46.853580+0000","last_clean":"2024-01-16T14:54:46.853580+0000","last_became_active":"2024-01-16T14:54:45.873897+0000","last_became_peered":"2024-01-16T14:54:45.873897+0000","last_unstale":"2024-01-16T14:54:46.853580+0000","last_undegraded":"2024-01-16T14:54:46.853580+0000","last_fullsized":"2024-01-16T14:54:46.853580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262205+0000","last_change":"2024-01-16T14:54:44.860218+0000","last_active":"2024-01-16T14:54:47.262205+0000","last_peered":"2024-01-16T14:54:47.262205+0000","last_clean":"2024-01-16T14:54:47.262205+0000","last_became_active":"2024-01-16T14:54:44.859920+0000","last_became_peered":"2024-01-16T14:54:44.859920+0000","last_unstale":"2024-01-16T14:54:47.262205+0000","last_undegraded":"2024-01-16T14:54:47.262205+0000","last_fullsized":"2024-01-16T14:54:47.262205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262345+0000","last_change":"2024-01-16T14:54:45.875405+0000","last_active":"2024-01-16T14:54:47.262345+0000","last_peered":"2024-01-16T14:54:47.262345+0000","last_clean":"2024-01-16T14:54:47.262345+0000","last_became_active":"2024-01-16T14:54:45.874428+0000","last_became_peered":"2024-01-16T14:54:45.874428+0000","last_unstale":"2024-01-16T14:54:47.262345+0000","last_undegraded":"2024-01-16T14:54:47.262345+0000","last_fullsized":"2024-01-16T14:54:47.262345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:06:27.912278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262076+0000","last_change":"2024-01-16T14:54:44.857601+0000","last_active":"2024-01-16T14:54:47.262076+0000","last_peered":"2024-01-16T14:54:47.262076+0000","last_clean":"2024-01-16T14:54:47.262076+0000","last_became_active":"2024-01-16T14:54:44.857282+0000","last_became_peered":"2024-01-16T14:54:44.857282+0000","last_unstale":"2024-01-16T14:54:47.262076+0000","last_undegraded":"2024-01-16T14:54:47.262076+0000","last_fullsized":"2024-01-16T14:54:47.262076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:52.733601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853602+0000","last_change":"2024-01-16T14:54:45.873626+0000","last_active":"2024-01-16T14:54:46.853602+0000","last_peered":"2024-01-16T14:54:46.853602+0000","last_clean":"2024-01-16T14:54:46.853602+0000","last_became_active":"2024-01-16T14:54:45.872632+0000","last_became_peered":"2024-01-16T14:54:45.872632+0000","last_unstale":"2024-01-16T14:54:46.853602+0000","last_undegraded":"2024-01-16T14:54:46.853602+0000","last_fullsized":"2024-01-16T14:54:46.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261375+0000","last_change":"2024-01-16T14:54:44.852946+0000","last_active":"2024-01-16T14:54:47.261375+0000","last_peered":"2024-01-16T14:54:47.261375+0000","last_clean":"2024-01-16T14:54:47.261375+0000","last_became_active":"2024-01-16T14:54:44.852682+0000","last_became_peered":"2024-01-16T14:54:44.852682+0000","last_unstale":"2024-01-16T14:54:47.261375+0000","last_undegraded":"2024-01-16T14:54:47.261375+0000","last_fullsized":"2024-01-16T14:54:47.261375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261947+0000","last_change":"2024-01-16T14:54:45.877152+0000","last_active":"2024-01-16T14:54:47.261947+0000","last_peered":"2024-01-16T14:54:47.261947+0000","last_clean":"2024-01-16T14:54:47.261947+0000","last_became_active":"2024-01-16T14:54:45.876716+0000","last_became_peered":"2024-01-16T14:54:45.876716+0000","last_unstale":"2024-01-16T14:54:47.261947+0000","last_undegraded":"2024-01-16T14:54:47.261947+0000","last_fullsized":"2024-01-16T14:54:47.261947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261401+0000","last_change":"2024-01-16T14:54:44.857725+0000","last_active":"2024-01-16T14:54:47.261401+0000","last_peered":"2024-01-16T14:54:47.261401+0000","last_clean":"2024-01-16T14:54:47.261401+0000","last_became_active":"2024-01-16T14:54:44.857273+0000","last_became_peered":"2024-01-16T14:54:44.857273+0000","last_unstale":"2024-01-16T14:54:47.261401+0000","last_undegraded":"2024-01-16T14:54:47.261401+0000","last_fullsized":"2024-01-16T14:54:47.261401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853385+0000","last_change":"2024-01-16T14:54:45.868095+0000","last_active":"2024-01-16T14:54:46.853385+0000","last_peered":"2024-01-16T14:54:46.853385+0000","last_clean":"2024-01-16T14:54:46.853385+0000","last_became_active":"2024-01-16T14:54:45.867008+0000","last_became_peered":"2024-01-16T14:54:45.867008+0000","last_unstale":"2024-01-16T14:54:46.853385+0000","last_undegraded":"2024-01-16T14:54:46.853385+0000","last_fullsized":"2024-01-16T14:54:46.853385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:50:57.724917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855353+0000","last_change":"2024-01-16T14:54:44.853674+0000","last_active":"2024-01-16T14:54:46.855353+0000","last_peered":"2024-01-16T14:54:46.855353+0000","last_clean":"2024-01-16T14:54:46.855353+0000","last_became_active":"2024-01-16T14:54:44.853079+0000","last_became_peered":"2024-01-16T14:54:44.853079+0000","last_unstale":"2024-01-16T14:54:46.855353+0000","last_undegraded":"2024-01-16T14:54:46.855353+0000","last_fullsized":"2024-01-16T14:54:46.855353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:02:31.747933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853861+0000","last_change":"2024-01-16T14:54:45.875678+0000","last_active":"2024-01-16T14:54:46.853861+0000","last_peered":"2024-01-16T14:54:46.853861+0000","last_clean":"2024-01-16T14:54:46.853861+0000","last_became_active":"2024-01-16T14:54:45.875400+0000","last_became_peered":"2024-01-16T14:54:45.875400+0000","last_unstale":"2024-01-16T14:54:46.853861+0000","last_undegraded":"2024-01-16T14:54:46.853861+0000","last_fullsized":"2024-01-16T14:54:46.853861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:25:20.146674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262644+0000","last_change":"2024-01-16T14:54:44.858592+0000","last_active":"2024-01-16T14:54:47.262644+0000","last_peered":"2024-01-16T14:54:47.262644+0000","last_clean":"2024-01-16T14:54:47.262644+0000","last_became_active":"2024-01-16T14:54:44.858275+0000","last_became_peered":"2024-01-16T14:54:44.858275+0000","last_unstale":"2024-01-16T14:54:47.262644+0000","last_undegraded":"2024-01-16T14:54:47.262644+0000","last_fullsized":"2024-01-16T14:54:47.262644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262293+0000","last_change":"2024-01-16T14:54:45.877869+0000","last_active":"2024-01-16T14:54:47.262293+0000","last_peered":"2024-01-16T14:54:47.262293+0000","last_clean":"2024-01-16T14:54:47.262293+0000","last_became_active":"2024-01-16T14:54:45.876905+0000","last_became_peered":"2024-01-16T14:54:45.876905+0000","last_unstale":"2024-01-16T14:54:47.262293+0000","last_undegraded":"2024-01-16T14:54:47.262293+0000","last_fullsized":"2024-01-16T14:54:47.262293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:47:51.438094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261976+0000","last_change":"2024-01-16T14:54:44.855986+0000","last_active":"2024-01-16T14:54:47.261976+0000","last_peered":"2024-01-16T14:54:47.261976+0000","last_clean":"2024-01-16T14:54:47.261976+0000","last_became_active":"2024-01-16T14:54:44.855690+0000","last_became_peered":"2024-01-16T14:54:44.855690+0000","last_unstale":"2024-01-16T14:54:47.261976+0000","last_undegraded":"2024-01-16T14:54:47.261976+0000","last_fullsized":"2024-01-16T14:54:47.261976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:33:34.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855013+0000","last_change":"2024-01-16T14:54:45.878441+0000","last_active":"2024-01-16T14:54:46.855013+0000","last_peered":"2024-01-16T14:54:46.855013+0000","last_clean":"2024-01-16T14:54:46.855013+0000","last_became_active":"2024-01-16T14:54:45.878011+0000","last_became_peered":"2024-01-16T14:54:45.878011+0000","last_unstale":"2024-01-16T14:54:46.855013+0000","last_undegraded":"2024-01-16T14:54:46.855013+0000","last_fullsized":"2024-01-16T14:54:46.855013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:05.890144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261593+0000","last_change":"2024-01-16T14:54:45.869056+0000","last_active":"2024-01-16T14:54:47.261593+0000","last_peered":"2024-01-16T14:54:47.261593+0000","last_clean":"2024-01-16T14:54:47.261593+0000","last_became_active":"2024-01-16T14:54:45.868748+0000","last_became_peered":"2024-01-16T14:54:45.868748+0000","last_unstale":"2024-01-16T14:54:47.261593+0000","last_undegraded":"2024-01-16T14:54:47.261593+0000","last_fullsized":"2024-01-16T14:54:47.261593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261832+0000","last_change":"2024-01-16T14:54:44.853961+0000","last_active":"2024-01-16T14:54:47.261832+0000","last_peered":"2024-01-16T14:54:47.261832+0000","last_clean":"2024-01-16T14:54:47.261832+0000","last_became_active":"2024-01-16T14:54:44.853610+0000","last_became_peered":"2024-01-16T14:54:44.853610+0000","last_unstale":"2024-01-16T14:54:47.261832+0000","last_undegraded":"2024-01-16T14:54:47.261832+0000","last_fullsized":"2024-01-16T14:54:47.261832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853929+0000","last_change":"2024-01-16T14:54:45.872254+0000","last_active":"2024-01-16T14:54:46.853929+0000","last_peered":"2024-01-16T14:54:46.853929+0000","last_clean":"2024-01-16T14:54:46.853929+0000","last_became_active":"2024-01-16T14:54:45.870597+0000","last_became_peered":"2024-01-16T14:54:45.870597+0000","last_unstale":"2024-01-16T14:54:46.853929+0000","last_undegraded":"2024-01-16T14:54:46.853929+0000","last_fullsized":"2024-01-16T14:54:46.853929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261437+0000","last_change":"2024-01-16T14:54:44.854781+0000","last_active":"2024-01-16T14:54:47.261437+0000","last_peered":"2024-01-16T14:54:47.261437+0000","last_clean":"2024-01-16T14:54:47.261437+0000","last_became_active":"2024-01-16T14:54:44.854461+0000","last_became_peered":"2024-01-16T14:54:44.854461+0000","last_unstale":"2024-01-16T14:54:47.261437+0000","last_undegraded":"2024-01-16T14:54:47.261437+0000","last_fullsized":"2024-01-16T14:54:47.261437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-16T14:54:45.859524+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:54:45.859524+0000","last_peered":"2024-01-16T14:54:45.859524+0000","last_clean":"2024-01-16T14:54:45.859524+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:54:45.859524+0000","last_undegraded":"2024-01-16T14:54:45.859524+0000","last_fullsized":"2024-01-16T14:54:45.859524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261727+0000","last_change":"2024-01-16T14:54:45.875199+0000","last_active":"2024-01-16T14:54:47.261727+0000","last_peered":"2024-01-16T14:54:47.261727+0000","last_clean":"2024-01-16T14:54:47.261727+0000","last_became_active":"2024-01-16T14:54:45.874049+0000","last_became_peered":"2024-01-16T14:54:45.874049+0000","last_unstale":"2024-01-16T14:54:47.261727+0000","last_undegraded":"2024-01-16T14:54:47.261727+0000","last_fullsized":"2024-01-16T14:54:47.261727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:29.976614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853314+0000","last_change":"2024-01-16T14:54:44.857691+0000","last_active":"2024-01-16T14:54:46.853314+0000","last_peered":"2024-01-16T14:54:46.853314+0000","last_clean":"2024-01-16T14:54:46.853314+0000","last_became_active":"2024-01-16T14:54:44.857407+0000","last_became_peered":"2024-01-16T14:54:44.857407+0000","last_unstale":"2024-01-16T14:54:46.853314+0000","last_undegraded":"2024-01-16T14:54:46.853314+0000","last_fullsized":"2024-01-16T14:54:46.853314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:18.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853856+0000","last_change":"2024-01-16T14:54:44.853542+0000","last_active":"2024-01-16T14:54:46.853856+0000","last_peered":"2024-01-16T14:54:46.853856+0000","last_clean":"2024-01-16T14:54:46.853856+0000","last_became_active":"2024-01-16T14:54:44.853273+0000","last_became_peered":"2024-01-16T14:54:44.853273+0000","last_unstale":"2024-01-16T14:54:46.853856+0000","last_undegraded":"2024-01-16T14:54:46.853856+0000","last_fullsized":"2024-01-16T14:54:46.853856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:56.322689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262718+0000","last_change":"2024-01-16T14:54:45.868977+0000","last_active":"2024-01-16T14:54:47.262718+0000","last_peered":"2024-01-16T14:54:47.262718+0000","last_clean":"2024-01-16T14:54:47.262718+0000","last_became_active":"2024-01-16T14:54:45.868675+0000","last_became_peered":"2024-01-16T14:54:45.868675+0000","last_unstale":"2024-01-16T14:54:47.262718+0000","last_undegraded":"2024-01-16T14:54:47.262718+0000","last_fullsized":"2024-01-16T14:54:47.262718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:48.406590+0000","last_change":"2024-01-16T14:54:44.855466+0000","last_active":"2024-01-16T14:54:48.406590+0000","last_peered":"2024-01-16T14:54:48.406590+0000","last_clean":"2024-01-16T14:54:48.406590+0000","last_became_active":"2024-01-16T14:54:44.855144+0000","last_became_peered":"2024-01-16T14:54:44.855144+0000","last_unstale":"2024-01-16T14:54:48.406590+0000","last_undegraded":"2024-01-16T14:54:48.406590+0000","last_fullsized":"2024-01-16T14:54:48.406590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854544+0000","last_change":"2024-01-16T14:54:45.875954+0000","last_active":"2024-01-16T14:54:46.854544+0000","last_peered":"2024-01-16T14:54:46.854544+0000","last_clean":"2024-01-16T14:54:46.854544+0000","last_became_active":"2024-01-16T14:54:45.875635+0000","last_became_peered":"2024-01-16T14:54:45.875635+0000","last_unstale":"2024-01-16T14:54:46.854544+0000","last_undegraded":"2024-01-16T14:54:46.854544+0000","last_fullsized":"2024-01-16T14:54:46.854544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262344+0000","last_change":"2024-01-16T14:54:44.858826+0000","last_active":"2024-01-16T14:54:47.262344+0000","last_peered":"2024-01-16T14:54:47.262344+0000","last_clean":"2024-01-16T14:54:47.262344+0000","last_became_active":"2024-01-16T14:54:44.858528+0000","last_became_peered":"2024-01-16T14:54:44.858528+0000","last_unstale":"2024-01-16T14:54:47.262344+0000","last_undegraded":"2024-01-16T14:54:47.262344+0000","last_fullsized":"2024-01-16T14:54:47.262344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:59:14.082325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262074+0000","last_change":"2024-01-16T14:54:45.872205+0000","last_active":"2024-01-16T14:54:47.262074+0000","last_peered":"2024-01-16T14:54:47.262074+0000","last_clean":"2024-01-16T14:54:47.262074+0000","last_became_active":"2024-01-16T14:54:45.871893+0000","last_became_peered":"2024-01-16T14:54:45.871893+0000","last_unstale":"2024-01-16T14:54:47.262074+0000","last_undegraded":"2024-01-16T14:54:47.262074+0000","last_fullsized":"2024-01-16T14:54:47.262074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853147+0000","last_change":"2024-01-16T14:54:44.853929+0000","last_active":"2024-01-16T14:54:46.853147+0000","last_peered":"2024-01-16T14:54:46.853147+0000","last_clean":"2024-01-16T14:54:46.853147+0000","last_became_active":"2024-01-16T14:54:44.853655+0000","last_became_peered":"2024-01-16T14:54:44.853655+0000","last_unstale":"2024-01-16T14:54:46.853147+0000","last_undegraded":"2024-01-16T14:54:46.853147+0000","last_fullsized":"2024-01-16T14:54:46.853147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.855031+0000","last_change":"2024-01-16T14:54:45.870957+0000","last_active":"2024-01-16T14:54:46.855031+0000","last_peered":"2024-01-16T14:54:46.855031+0000","last_clean":"2024-01-16T14:54:46.855031+0000","last_became_active":"2024-01-16T14:54:45.870672+0000","last_became_peered":"2024-01-16T14:54:45.870672+0000","last_unstale":"2024-01-16T14:54:46.855031+0000","last_undegraded":"2024-01-16T14:54:46.855031+0000","last_fullsized":"2024-01-16T14:54:46.855031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:28:19.153610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261884+0000","last_change":"2024-01-16T14:54:44.859174+0000","last_active":"2024-01-16T14:54:47.261884+0000","last_peered":"2024-01-16T14:54:47.261884+0000","last_clean":"2024-01-16T14:54:47.261884+0000","last_became_active":"2024-01-16T14:54:44.858845+0000","last_became_peered":"2024-01-16T14:54:44.858845+0000","last_unstale":"2024-01-16T14:54:47.261884+0000","last_undegraded":"2024-01-16T14:54:47.261884+0000","last_fullsized":"2024-01-16T14:54:47.261884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:52:38.607728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853729+0000","last_change":"2024-01-16T14:54:45.865608+0000","last_active":"2024-01-16T14:54:46.853729+0000","last_peered":"2024-01-16T14:54:46.853729+0000","last_clean":"2024-01-16T14:54:46.853729+0000","last_became_active":"2024-01-16T14:54:45.864856+0000","last_became_peered":"2024-01-16T14:54:45.864856+0000","last_unstale":"2024-01-16T14:54:46.853729+0000","last_undegraded":"2024-01-16T14:54:46.853729+0000","last_fullsized":"2024-01-16T14:54:46.853729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262559+0000","last_change":"2024-01-16T14:54:44.857846+0000","last_active":"2024-01-16T14:54:47.262559+0000","last_peered":"2024-01-16T14:54:47.262559+0000","last_clean":"2024-01-16T14:54:47.262559+0000","last_became_active":"2024-01-16T14:54:44.857244+0000","last_became_peered":"2024-01-16T14:54:44.857244+0000","last_unstale":"2024-01-16T14:54:47.262559+0000","last_undegraded":"2024-01-16T14:54:47.262559+0000","last_fullsized":"2024-01-16T14:54:47.262559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854055+0000","last_change":"2024-01-16T14:54:45.877240+0000","last_active":"2024-01-16T14:54:46.854055+0000","last_peered":"2024-01-16T14:54:46.854055+0000","last_clean":"2024-01-16T14:54:46.854055+0000","last_became_active":"2024-01-16T14:54:45.876921+0000","last_became_peered":"2024-01-16T14:54:45.876921+0000","last_unstale":"2024-01-16T14:54:46.854055+0000","last_undegraded":"2024-01-16T14:54:46.854055+0000","last_fullsized":"2024-01-16T14:54:46.854055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853720+0000","last_change":"2024-01-16T14:54:44.841886+0000","last_active":"2024-01-16T14:54:46.853720+0000","last_peered":"2024-01-16T14:54:46.853720+0000","last_clean":"2024-01-16T14:54:46.853720+0000","last_became_active":"2024-01-16T14:54:44.841590+0000","last_became_peered":"2024-01-16T14:54:44.841590+0000","last_unstale":"2024-01-16T14:54:46.853720+0000","last_undegraded":"2024-01-16T14:54:46.853720+0000","last_fullsized":"2024-01-16T14:54:46.853720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261806+0000","last_change":"2024-01-16T14:54:45.861293+0000","last_active":"2024-01-16T14:54:47.261806+0000","last_peered":"2024-01-16T14:54:47.261806+0000","last_clean":"2024-01-16T14:54:47.261806+0000","last_became_active":"2024-01-16T14:54:45.860966+0000","last_became_peered":"2024-01-16T14:54:45.860966+0000","last_unstale":"2024-01-16T14:54:47.261806+0000","last_undegraded":"2024-01-16T14:54:47.261806+0000","last_fullsized":"2024-01-16T14:54:47.261806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:16:01.958056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262495+0000","last_change":"2024-01-16T14:54:44.859596+0000","last_active":"2024-01-16T14:54:47.262495+0000","last_peered":"2024-01-16T14:54:47.262495+0000","last_clean":"2024-01-16T14:54:47.262495+0000","last_became_active":"2024-01-16T14:54:44.859150+0000","last_became_peered":"2024-01-16T14:54:44.859150+0000","last_unstale":"2024-01-16T14:54:47.262495+0000","last_undegraded":"2024-01-16T14:54:47.262495+0000","last_fullsized":"2024-01-16T14:54:47.262495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:19:04.884869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853762+0000","last_change":"2024-01-16T14:54:45.884933+0000","last_active":"2024-01-16T14:54:46.853762+0000","last_peered":"2024-01-16T14:54:46.853762+0000","last_clean":"2024-01-16T14:54:46.853762+0000","last_became_active":"2024-01-16T14:54:45.884688+0000","last_became_peered":"2024-01-16T14:54:45.884688+0000","last_unstale":"2024-01-16T14:54:46.853762+0000","last_undegraded":"2024-01-16T14:54:46.853762+0000","last_fullsized":"2024-01-16T14:54:46.853762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:43:46.825323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262635+0000","last_change":"2024-01-16T14:54:44.852405+0000","last_active":"2024-01-16T14:54:47.262635+0000","last_peered":"2024-01-16T14:54:47.262635+0000","last_clean":"2024-01-16T14:54:47.262635+0000","last_became_active":"2024-01-16T14:54:44.852011+0000","last_became_peered":"2024-01-16T14:54:44.852011+0000","last_unstale":"2024-01-16T14:54:47.262635+0000","last_undegraded":"2024-01-16T14:54:47.262635+0000","last_fullsized":"2024-01-16T14:54:47.262635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:54:56.086784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261800+0000","last_change":"2024-01-16T14:54:45.883630+0000","last_active":"2024-01-16T14:54:47.261800+0000","last_peered":"2024-01-16T14:54:47.261800+0000","last_clean":"2024-01-16T14:54:47.261800+0000","last_became_active":"2024-01-16T14:54:45.882901+0000","last_became_peered":"2024-01-16T14:54:45.882901+0000","last_unstale":"2024-01-16T14:54:47.261800+0000","last_undegraded":"2024-01-16T14:54:47.261800+0000","last_fullsized":"2024-01-16T14:54:47.261800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:40:04.562279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262954+0000","last_change":"2024-01-16T14:54:44.853825+0000","last_active":"2024-01-16T14:54:47.262954+0000","last_peered":"2024-01-16T14:54:47.262954+0000","last_clean":"2024-01-16T14:54:47.262954+0000","last_became_active":"2024-01-16T14:54:44.853509+0000","last_became_peered":"2024-01-16T14:54:44.853509+0000","last_unstale":"2024-01-16T14:54:47.262954+0000","last_undegraded":"2024-01-16T14:54:47.262954+0000","last_fullsized":"2024-01-16T14:54:47.262954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:43.826819+0000","last_clean_scrub_stamp":"2024-01-16T14:54:43.826819+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:05:42.365051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853960+0000","last_change":"2024-01-16T14:54:45.878029+0000","last_active":"2024-01-16T14:54:46.853960+0000","last_peered":"2024-01-16T14:54:46.853960+0000","last_clean":"2024-01-16T14:54:46.853960+0000","last_became_active":"2024-01-16T14:54:45.877266+0000","last_became_peered":"2024-01-16T14:54:45.877266+0000","last_unstale":"2024-01-16T14:54:46.853960+0000","last_undegraded":"2024-01-16T14:54:46.853960+0000","last_fullsized":"2024-01-16T14:54:46.853960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:47:32.931018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261228+0000","last_change":"2024-01-16T14:54:45.876962+0000","last_active":"2024-01-16T14:54:47.261228+0000","last_peered":"2024-01-16T14:54:47.261228+0000","last_clean":"2024-01-16T14:54:47.261228+0000","last_became_active":"2024-01-16T14:54:45.876358+0000","last_became_peered":"2024-01-16T14:54:45.876358+0000","last_unstale":"2024-01-16T14:54:47.261228+0000","last_undegraded":"2024-01-16T14:54:47.261228+0000","last_fullsized":"2024-01-16T14:54:47.261228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:53.738876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261492+0000","last_change":"2024-01-16T14:54:45.874788+0000","last_active":"2024-01-16T14:54:47.261492+0000","last_peered":"2024-01-16T14:54:47.261492+0000","last_clean":"2024-01-16T14:54:47.261492+0000","last_became_active":"2024-01-16T14:54:45.874453+0000","last_became_peered":"2024-01-16T14:54:45.874453+0000","last_unstale":"2024-01-16T14:54:47.261492+0000","last_undegraded":"2024-01-16T14:54:47.261492+0000","last_fullsized":"2024-01-16T14:54:47.261492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:12:18.274045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261754+0000","last_change":"2024-01-16T14:54:45.877374+0000","last_active":"2024-01-16T14:54:47.261754+0000","last_peered":"2024-01-16T14:54:47.261754+0000","last_clean":"2024-01-16T14:54:47.261754+0000","last_became_active":"2024-01-16T14:54:45.877118+0000","last_became_peered":"2024-01-16T14:54:45.877118+0000","last_unstale":"2024-01-16T14:54:47.261754+0000","last_undegraded":"2024-01-16T14:54:47.261754+0000","last_fullsized":"2024-01-16T14:54:47.261754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:11:35.941820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262220+0000","last_change":"2024-01-16T14:54:45.875476+0000","last_active":"2024-01-16T14:54:47.262220+0000","last_peered":"2024-01-16T14:54:47.262220+0000","last_clean":"2024-01-16T14:54:47.262220+0000","last_became_active":"2024-01-16T14:54:45.874303+0000","last_became_peered":"2024-01-16T14:54:45.874303+0000","last_unstale":"2024-01-16T14:54:47.262220+0000","last_undegraded":"2024-01-16T14:54:47.262220+0000","last_fullsized":"2024-01-16T14:54:47.262220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:34:34.073528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854023+0000","last_change":"2024-01-16T14:54:45.866307+0000","last_active":"2024-01-16T14:54:46.854023+0000","last_peered":"2024-01-16T14:54:46.854023+0000","last_clean":"2024-01-16T14:54:46.854023+0000","last_became_active":"2024-01-16T14:54:45.865419+0000","last_became_peered":"2024-01-16T14:54:45.865419+0000","last_unstale":"2024-01-16T14:54:46.854023+0000","last_undegraded":"2024-01-16T14:54:46.854023+0000","last_fullsized":"2024-01-16T14:54:46.854023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262723+0000","last_change":"2024-01-16T14:54:45.866420+0000","last_active":"2024-01-16T14:54:47.262723+0000","last_peered":"2024-01-16T14:54:47.262723+0000","last_clean":"2024-01-16T14:54:47.262723+0000","last_became_active":"2024-01-16T14:54:45.866022+0000","last_became_peered":"2024-01-16T14:54:45.866022+0000","last_unstale":"2024-01-16T14:54:47.262723+0000","last_undegraded":"2024-01-16T14:54:47.262723+0000","last_fullsized":"2024-01-16T14:54:47.262723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:31:12.899684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262181+0000","last_change":"2024-01-16T14:54:45.875523+0000","last_active":"2024-01-16T14:54:47.262181+0000","last_peered":"2024-01-16T14:54:47.262181+0000","last_clean":"2024-01-16T14:54:47.262181+0000","last_became_active":"2024-01-16T14:54:45.875252+0000","last_became_peered":"2024-01-16T14:54:45.875252+0000","last_unstale":"2024-01-16T14:54:47.262181+0000","last_undegraded":"2024-01-16T14:54:47.262181+0000","last_fullsized":"2024-01-16T14:54:47.262181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:16:51.879812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.854401+0000","last_change":"2024-01-16T14:54:45.865807+0000","last_active":"2024-01-16T14:54:46.854401+0000","last_peered":"2024-01-16T14:54:46.854401+0000","last_clean":"2024-01-16T14:54:46.854401+0000","last_became_active":"2024-01-16T14:54:45.865257+0000","last_became_peered":"2024-01-16T14:54:45.865257+0000","last_unstale":"2024-01-16T14:54:46.854401+0000","last_undegraded":"2024-01-16T14:54:46.854401+0000","last_fullsized":"2024-01-16T14:54:46.854401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261310+0000","last_change":"2024-01-16T14:54:45.883032+0000","last_active":"2024-01-16T14:54:47.261310+0000","last_peered":"2024-01-16T14:54:47.261310+0000","last_clean":"2024-01-16T14:54:47.261310+0000","last_became_active":"2024-01-16T14:54:45.882768+0000","last_became_peered":"2024-01-16T14:54:45.882768+0000","last_unstale":"2024-01-16T14:54:47.261310+0000","last_undegraded":"2024-01-16T14:54:47.261310+0000","last_fullsized":"2024-01-16T14:54:47.261310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:54:40.602461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:46.853768+0000","last_change":"2024-01-16T14:54:45.866167+0000","last_active":"2024-01-16T14:54:46.853768+0000","last_peered":"2024-01-16T14:54:46.853768+0000","last_clean":"2024-01-16T14:54:46.853768+0000","last_became_active":"2024-01-16T14:54:45.865393+0000","last_became_peered":"2024-01-16T14:54:45.865393+0000","last_unstale":"2024-01-16T14:54:46.853768+0000","last_undegraded":"2024-01-16T14:54:46.853768+0000","last_fullsized":"2024-01-16T14:54:46.853768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261572+0000","last_change":"2024-01-16T14:54:45.882612+0000","last_active":"2024-01-16T14:54:47.261572+0000","last_peered":"2024-01-16T14:54:47.261572+0000","last_clean":"2024-01-16T14:54:47.261572+0000","last_became_active":"2024-01-16T14:54:45.882301+0000","last_became_peered":"2024-01-16T14:54:45.882301+0000","last_unstale":"2024-01-16T14:54:47.261572+0000","last_undegraded":"2024-01-16T14:54:47.261572+0000","last_fullsized":"2024-01-16T14:54:47.261572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:02:25.547345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.262090+0000","last_change":"2024-01-16T14:54:45.876292+0000","last_active":"2024-01-16T14:54:47.262090+0000","last_peered":"2024-01-16T14:54:47.262090+0000","last_clean":"2024-01-16T14:54:47.262090+0000","last_became_active":"2024-01-16T14:54:45.875995+0000","last_became_peered":"2024-01-16T14:54:45.875995+0000","last_unstale":"2024-01-16T14:54:47.262090+0000","last_undegraded":"2024-01-16T14:54:47.262090+0000","last_fullsized":"2024-01-16T14:54:47.262090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:26:57.327965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-16T14:54:47.261512+0000","last_change":"2024-01-16T14:54:45.866707+0000","last_active":"2024-01-16T14:54:47.261512+0000","last_peered":"2024-01-16T14:54:47.261512+0000","last_clean":"2024-01-16T14:54:47.261512+0000","last_became_active":"2024-01-16T14:54:45.866333+0000","last_became_peered":"2024-01-16T14:54:45.866333+0000","last_unstale":"2024-01-16T14:54:47.261512+0000","last_undegraded":"2024-01-16T14:54:47.261512+0000","last_fullsized":"2024-01-16T14:54:47.261512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:44.832094+0000","last_clean_scrub_stamp":"2024-01-16T14:54:44.832094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:06:55.576570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":76574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:54:55.575 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-16T14:54:55.575 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-16T14:54:55.575 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-16T14:54:55.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-16T14:54:55.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.767+0000 7f86527a5700 1 Processor -- start 2024-01-16T14:54:55.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.767+0000 7f86527a5700 1 -- start start 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f86527a5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f864c15e000 0x7f864c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f86527a5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c0d0ae0 0x7f864c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f86527a5700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c0cfd60 con 0x7f864c0d0f60 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f86527a5700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c0cf4b0 con 0x7f864c15e000 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f86527a5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c06f430 con 0x7f864c0d0ae0 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f864bfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f864c15e000 0x7f864c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f8650d42700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f864c0d0f60 0x7f864c0cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58588/0 (socket says 172.21.15.26:58588) 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f8650d42700 1 -- 172.21.15.26:0/2131697670 learned_addr learned my addr 172.21.15.26:0/2131697670 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:55.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.768+0000 7f864b7fe700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c0d0ae0 0x7f864c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864bfff700 1 -- 172.21.15.26:0/2131697670 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c0d0ae0 msgr2=0x7f864c06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864bfff700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c0d0ae0 0x7f864c06f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864bfff700 1 -- 172.21.15.26:0/2131697670 >> v1:172.21.15.26:6789/0 conn(0x7f864c0d0f60 legacy=0x7f864c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864affd700 1 -- 172.21.15.26:0/2131697670 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284422093 0 0) 0x7f864c0cfd60 con 0x7f864c0d0f60 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864bfff700 1 -- 172.21.15.26:0/2131697670 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0cf1d0 con 0x7f864c15e000 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864bfff700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c15e000 0x7f864c06eef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f863c009840 tx=0x7f863c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a877ca0401b374 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864affd700 1 -- 172.21.15.26:0/2131697670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c011830 con 0x7f864c15e000 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864affd700 1 -- 172.21.15.26:0/2131697670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c011990 con 0x7f864c15e000 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.769+0000 7f864affd700 1 -- 172.21.15.26:0/2131697670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c011d20 con 0x7f864c15e000 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 -- 172.21.15.26:0/2131697670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c15e000 msgr2=0x7f864c06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c15e000 0x7f864c06eef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f863c009840 tx=0x7f863c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 -- 172.21.15.26:0/2131697670 shutdown_connections 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c15e000 0x7f864c06eef0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 --2- 172.21.15.26:0/2131697670 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c0d0ae0 0x7f864c06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 -- 172.21.15.26:0/2131697670 >> 172.21.15.26:0/2131697670 conn(0x7f864c0b3720 msgr2=0x7f864c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:55.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 -- 172.21.15.26:0/2131697670 shutdown_connections 2024-01-16T14:54:55.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.770+0000 7f86527a5700 1 -- 172.21.15.26:0/2131697670 wait complete. 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.771+0000 7f86527a5700 1 Processor -- start 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.771+0000 7f86527a5700 1 -- start start 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f864c0d0f60 0x7f864c1608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c15e000 0x7f864c14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c159560 con 0x7f864c0d0ae0 2024-01-16T14:54:55.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c15e480 con 0x7f864c0d0f60 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f86527a5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c150520 con 0x7f864c15e000 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55364/0 (socket says 172.21.15.26:55364) 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 -- 172.21.15.26:0/1065013707 learned_addr learned my addr 172.21.15.26:0/1065013707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864b7fe700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f864c0d0f60 0x7f864c1608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f8650d42700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c15e000 0x7f864c14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 -- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c15e000 msgr2=0x7f864c14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c15e000 0x7f864c14ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 -- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f864c0d0f60 msgr2=0x7f864c1608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f864c0d0f60 0x7f864c1608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.772+0000 7f864bfff700 1 -- 172.21.15.26:0/1065013707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0094a0 con 0x7f864c0d0ae0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f864bfff700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f863c009790 tx=0x7f863c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0036a0 con 0x7f864c0d0ae0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c01e070 con 0x7f864c0d0ae0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c0c6170 con 0x7f864c0d0ae0 2024-01-16T14:54:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c003a20 con 0x7f864c0d0ae0 2024-01-16T14:54:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.773+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c0c6730 con 0x7f864c0d0ae0 2024-01-16T14:54:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.774+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c160de0 con 0x7f864c0d0ae0 2024-01-16T14:54:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.774+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f863c003b80 con 0x7f864c0d0ae0 2024-01-16T14:54:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.775+0000 7f8648ff9700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 0x7f863806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.775+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f863c08ac40 con 0x7f864c0d0ae0 2024-01-16T14:54:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.775+0000 7f864b7fe700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 0x7f863806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.776+0000 7f864b7fe700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 0x7f863806d810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f864c0c60a0 tx=0x7f8640008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:55.780+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f863c055b40 con 0x7f864c0d0ae0 2024-01-16T14:54:56.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.037+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f864c068670 con 0x7f864c0d0ae0 2024-01-16T14:54:56.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.038+0000 7f8648ff9700 1 -- 172.21.15.26:0/1065013707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f863c059160 con 0x7f864c0d0ae0 2024-01-16T14:54:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:56.021 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-16T14:54:56.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 msgr2=0x7f863806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 0x7f863806d810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f864c0c60a0 tx=0x7f8640008040 comp rx=0 tx=0).stop 2024-01-16T14:54:56.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 msgr2=0x7f864c159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f863c009790 tx=0x7f863c004750 comp rx=0 tx=0).stop 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 shutdown_connections 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f864c15e000 0x7f864c14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f863806b350 0x7f863806d810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f864c0d0ae0 0x7f864c159760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 --2- 172.21.15.26:0/1065013707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f864c0d0f60 0x7f864c1608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 >> 172.21.15.26:0/1065013707 conn(0x7f864c0b3720 msgr2=0x7f864c0ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 shutdown_connections 2024-01-16T14:54:56.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.040+0000 7f86527a5700 1 -- 172.21.15.26:0/1065013707 wait complete. 2024-01-16T14:54:56.037 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-16T14:54:56.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:56.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.190+0000 7f223407a700 1 Processor -- start 2024-01-16T14:54:56.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.190+0000 7f223407a700 1 -- start start 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f223407a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f222c1448b0 0x7f222c06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f223407a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c144c90 con 0x7f222c06f990 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f223407a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0b4c80 con 0x7f222c0b44c0 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f223407a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0b4f60 con 0x7f222c1448b0 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2232617700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f222c06f990 0x7f222c0ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58594/0 (socket says 172.21.15.26:58594) 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2232617700 1 -- 172.21.15.26:0/264100753 learned_addr learned my addr 172.21.15.26:0/264100753 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2231e16700 1 --2- 172.21.15.26:0/264100753 >> v2:172.21.15.93:3300/0 conn(0x7f222c1448b0 0x7f222c06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2230e14700 1 -- 172.21.15.26:0/264100753 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782764747 0 0) 0x7f222c144c90 con 0x7f222c06f990 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2230e14700 1 -- 172.21.15.26:0/264100753 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22180035b0 con 0x7f222c06f990 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.191+0000 7f2231e16700 1 -- 172.21.15.26:0/264100753 >> v1:172.21.15.93:6789/0 conn(0x7f222c0b44c0 legacy=0x7f222c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2231e16700 1 -- 172.21.15.26:0/264100753 >> v1:172.21.15.26:6789/0 conn(0x7f222c06f990 legacy=0x7f222c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2231e16700 1 -- 172.21.15.26:0/264100753 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0b5240 con 0x7f222c1448b0 2024-01-16T14:54:56.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2231e16700 1 --2- 172.21.15.26:0/264100753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c1448b0 0x7f222c06ec70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f221c009840 tx=0x7f221c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4117b7a5f3fd4bf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2230e14700 1 -- 172.21.15.26:0/264100753 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c011830 con 0x7f222c1448b0 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2230e14700 1 -- 172.21.15.26:0/264100753 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c011990 con 0x7f222c1448b0 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f2230e14700 1 -- 172.21.15.26:0/264100753 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c011d00 con 0x7f222c1448b0 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f223407a700 1 -- 172.21.15.26:0/264100753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c1448b0 msgr2=0x7f222c06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.192+0000 7f223407a700 1 --2- 172.21.15.26:0/264100753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c1448b0 0x7f222c06ec70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f221c009840 tx=0x7f221c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.193+0000 7f223407a700 1 -- 172.21.15.26:0/264100753 shutdown_connections 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.193+0000 7f223407a700 1 --2- 172.21.15.26:0/264100753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c1448b0 0x7f222c06ec70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.193+0000 7f223407a700 1 -- 172.21.15.26:0/264100753 >> 172.21.15.26:0/264100753 conn(0x7f222c0d0bc0 msgr2=0x7f222c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.193+0000 7f223407a700 1 -- 172.21.15.26:0/264100753 shutdown_connections 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.193+0000 7f223407a700 1 -- 172.21.15.26:0/264100753 wait complete. 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.194+0000 7f223407a700 1 Processor -- start 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.194+0000 7f223407a700 1 -- start start 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 0x7f222c13fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f222c0b44c0 0x7f222c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c142620 con 0x7f222c1448b0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c141aa0 con 0x7f222c06f990 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f223407a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c141d80 con 0x7f222c0b44c0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2231e16700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 0x7f222c13fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55378/0 (socket says 172.21.15.26:55378) 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2231e16700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 0x7f222c13fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:37088/0 (socket says 172.21.15.26:37088) 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 -- 172.21.15.26:0/2161362131 learned_addr learned my addr 172.21.15.26:0/2161362131 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2231615700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f222c0b44c0 0x7f222c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 -- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f222c0b44c0 msgr2=0x7f222c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f222c0b44c0 0x7f222c06afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 -- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 msgr2=0x7f222c13fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 0x7f222c13fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 -- 172.21.15.26:0/2161362131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c0094a0 con 0x7f222c1448b0 2024-01-16T14:54:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.195+0000 7f2232617700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f222400cbb0 tx=0x7f222400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.196+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222400a9e0 con 0x7f222c1448b0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.196+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222400ab40 con 0x7f222c1448b0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.196+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c142060 con 0x7f222c1448b0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.196+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224012400 con 0x7f222c1448b0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.196+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c141730 con 0x7f222c1448b0 2024-01-16T14:54:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.197+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c1f4c10 con 0x7f222c1448b0 2024-01-16T14:54:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.197+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f222401b3b0 con 0x7f222c1448b0 2024-01-16T14:54:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.198+0000 7f22227fc700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 0x7f221006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.198+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2224089760 con 0x7f222c1448b0 2024-01-16T14:54:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.198+0000 7f2231e16700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 0x7f221006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.198+0000 7f2231e16700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 0x7f221006d810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.202+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f222c1f4c10 con 0x7f222c1448b0 2024-01-16T14:54:56.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.416+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222c1f4c10 con 0x7f222c1448b0 2024-01-16T14:54:56.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.417+0000 7f22227fc700 1 -- 172.21.15.26:0/2161362131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f222c1f4c10 con 0x7f222c1448b0 2024-01-16T14:54:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:56.401 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 msgr2=0x7f221006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 0x7f221006d810 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f221c000c00 tx=0x7f221c01c040 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 msgr2=0x7f222c06b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f222400cbb0 tx=0x7f222400cec0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 shutdown_connections 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f221006b350 0x7f221006d810 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f222c1448b0 0x7f222c06b510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f222c06f990 0x7f222c13fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 --2- 172.21.15.26:0/2161362131 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f222c0b44c0 0x7f222c06afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.419+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 >> 172.21.15.26:0/2161362131 conn(0x7f222c0d0bc0 msgr2=0x7f222c0cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.420+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 shutdown_connections 2024-01-16T14:54:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.420+0000 7f223407a700 1 -- 172.21.15.26:0/2161362131 wait complete. 2024-01-16T14:54:56.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:54:56.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:54:56.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.574+0000 7f5ea0c08700 1 Processor -- start 2024-01-16T14:54:56.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.574+0000 7f5ea0c08700 1 -- start start 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.575+0000 7f5ea0c08700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5e9c1457a0 0x7f5e9c0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.575+0000 7f5ea0c08700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.575+0000 7f5ea0c08700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9c0d04b0 con 0x7f5e9c06afd0 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.575+0000 7f5ea0c08700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9c160870 con 0x7f5e9c1457a0 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.575+0000 7f5ea0c08700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c0cf1b0 con 0x7f5e9c0cd470 2024-01-16T14:54:56.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e99d9b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5e9c1457a0 0x7f5e9c0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e99d9b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:59398/0 (socket says 172.21.15.26:59398) 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e99d9b700 1 -- 172.21.15.26:0/3481617643 learned_addr learned my addr 172.21.15.26:0/3481617643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 -- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 msgr2=0x7f5e9c160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 -- 172.21.15.26:0/3481617643 >> v1:172.21.15.93:6789/0 conn(0x7f5e9c06afd0 legacy=0x7f5e9c06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 -- 172.21.15.26:0/3481617643 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0cf490 con 0x7f5e9c1457a0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9959a700 1 -- 172.21.15.26:0/3481617643 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076316384 0 0) 0x7f5e9c0d04b0 con 0x7f5e9c06afd0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e99d9b700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.576+0000 7f5e9a59c700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c0cff70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009840 tx=0x7f5e8c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef76eefd14ba543 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5e9959a700 1 -- 172.21.15.26:0/3481617643 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c011830 con 0x7f5e9c1457a0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5e9959a700 1 -- 172.21.15.26:0/3481617643 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8c011990 con 0x7f5e9c1457a0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5e9959a700 1 -- 172.21.15.26:0/3481617643 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c011d20 con 0x7f5e9c1457a0 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c1457a0 msgr2=0x7f5e9c0cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c0cff70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c009840 tx=0x7f5e8c009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3481617643 shutdown_connections 2024-01-16T14:54:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c0cff70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3481617643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3481617643 >> 172.21.15.26:0/3481617643 conn(0x7f5e9c0d0aa0 msgr2=0x7f5e9c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3481617643 shutdown_connections 2024-01-16T14:54:56.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.577+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3481617643 wait complete. 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 Processor -- start 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 -- start start 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c06afd0 0x7f5e9c1f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c155300 con 0x7f5e9c1457a0 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c154e70 con 0x7f5e9c06afd0 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.579+0000 7f5ea0c08700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c154260 con 0x7f5e9c0cd470 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55386/0 (socket says 172.21.15.26:55386) 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 -- 172.21.15.26:0/3456782777 learned_addr learned my addr 172.21.15.26:0/3456782777 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e99d9b700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9a59c700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c06afd0 0x7f5e9c1f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 -- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 msgr2=0x7f5e9c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 -- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c06afd0 msgr2=0x7f5e9c1f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c06afd0 0x7f5e9c1f0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 -- 172.21.15.26:0/3456782777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90009550 con 0x7f5e9c1457a0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e9ad9d700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c760 tx=0x7f5e9400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94003e30 con 0x7f5e9c1457a0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94005070 con 0x7f5e9c1457a0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c0094a0 con 0x7f5e9c1457a0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.580+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94005400 con 0x7f5e9c1457a0 2024-01-16T14:54:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.581+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9c154820 con 0x7f5e9c1457a0 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.581+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9c155000 con 0x7f5e9c1457a0 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.582+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5e94005560 con 0x7f5e9c1457a0 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.582+0000 7f5e8affd700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 0x7f5e7c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.582+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e94089790 con 0x7f5e9c1457a0 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.583+0000 7f5e9a59c700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 0x7f5e7c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.583+0000 7f5e9a59c700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 0x7f5e7c06d840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.587+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e940545f0 con 0x7f5e9c1457a0 2024-01-16T14:54:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.794+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e9c075480 con 0x7f5e9c1457a0 2024-01-16T14:54:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.795+0000 7f5e8affd700 1 -- 172.21.15.26:0/3456782777 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e94057c10 con 0x7f5e9c1457a0 2024-01-16T14:54:56.778 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 msgr2=0x7f5e7c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 0x7f5e7c06d840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c00 tx=0x7f5e8c008030 comp rx=0 tx=0).stop 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 msgr2=0x7f5e9c156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400c760 tx=0x7f5e9400ca70 comp rx=0 tx=0).stop 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 shutdown_connections 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5e7c06b380 0x7f5e7c06d840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e9c1457a0 0x7f5e9c156b30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e9c06afd0 0x7f5e9c1f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 --2- 172.21.15.26:0/3456782777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e9c0cd470 0x7f5e9c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.797+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 >> 172.21.15.26:0/3456782777 conn(0x7f5e9c0d0aa0 msgr2=0x7f5e9c0cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.798+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 shutdown_connections 2024-01-16T14:54:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.798+0000 7f5ea0c08700 1 -- 172.21.15.26:0/3456782777 wait complete. 2024-01-16T14:54:56.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:54:56.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.952+0000 7f539dbf1700 1 Processor -- start 2024-01-16T14:54:56.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.952+0000 7f539dbf1700 1 -- start start 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f539dbf1700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f539815c1c0 0x7f53980cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f539dbf1700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53980cc670 con 0x7f53980b4240 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f539dbf1700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53980cf7a0 con 0x7f539815bde0 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f539dbf1700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53980cffb0 con 0x7f539815c1c0 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f5397fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53980b4240 0x7f539815fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58606/0 (socket says 172.21.15.26:58606) 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f5397fff700 1 -- 172.21.15.26:0/371296833 learned_addr learned my addr 172.21.15.26:0/371296833 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:56.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f538ffff700 1 --2- 172.21.15.26:0/371296833 >> v2:172.21.15.93:3300/0 conn(0x7f539815c1c0 0x7f53980cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755470043 0 0) 0x7f53980cc670 con 0x7f53980b4240 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53840035b0 con 0x7f53980b4240 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.953+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797477723 0 0) 0x7f53980cf7a0 con 0x7f539815bde0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53980cc670 con 0x7f539815bde0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f538ffff700 1 -- 172.21.15.26:0/371296833 >> v1:172.21.15.93:6789/0 conn(0x7f539815bde0 legacy=0x7f539806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f538ffff700 1 -- 172.21.15.26:0/371296833 >> v1:172.21.15.26:6789/0 conn(0x7f53980b4240 legacy=0x7f539815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f538ffff700 1 -- 172.21.15.26:0/371296833 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539806f3f0 con 0x7f539815c1c0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f538ffff700 1 --2- 172.21.15.26:0/371296833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f539815c1c0 0x7f53980cd0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5380009840 tx=0x7f5380009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a00839ba7a20b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380011830 con 0x7f539815c1c0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380011990 con 0x7f539815c1c0 2024-01-16T14:54:56.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.954+0000 7f5396ffd700 1 -- 172.21.15.26:0/371296833 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380011d20 con 0x7f539815c1c0 2024-01-16T14:54:56.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 -- 172.21.15.26:0/371296833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f539815c1c0 msgr2=0x7f53980cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 --2- 172.21.15.26:0/371296833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f539815c1c0 0x7f53980cd0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5380009840 tx=0x7f5380009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:56.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 -- 172.21.15.26:0/371296833 shutdown_connections 2024-01-16T14:54:56.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 --2- 172.21.15.26:0/371296833 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f539815c1c0 0x7f53980cd0a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 -- 172.21.15.26:0/371296833 >> 172.21.15.26:0/371296833 conn(0x7f53980b3720 msgr2=0x7f53980b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 -- 172.21.15.26:0/371296833 shutdown_connections 2024-01-16T14:54:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.955+0000 7f539dbf1700 1 -- 172.21.15.26:0/371296833 wait complete. 2024-01-16T14:54:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.956+0000 7f539dbf1700 1 Processor -- start 2024-01-16T14:54:56.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.956+0000 7f539dbf1700 1 -- start start 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 0x7f53980c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53980c4950 0x7f539815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539815eed0 con 0x7f53980b4240 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980c4e00 con 0x7f53980c4950 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f539dbf1700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539815e250 con 0x7f539815bde0 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55400/0 (socket says 172.21.15.26:55400) 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 -- 172.21.15.26:0/753624684 learned_addr learned my addr 172.21.15.26:0/753624684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:56.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f5397fff700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53980c4950 0x7f539815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f538ffff700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 0x7f53980c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 -- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 msgr2=0x7f53980c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 0x7f53980c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 -- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53980c4950 msgr2=0x7f539815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53980c4950 0x7f539815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 -- 172.21.15.26:0/753624684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800094a0 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f538ffff700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 0x7f53980c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.957+0000 7f53977fe700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f538800c520 tx=0x7f538800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.958+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880052d0 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.958+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5388005430 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.958+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880038a0 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.958+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539815e530 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.958+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539815eaf0 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.959+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5398075480 con 0x7f53980b4240 2024-01-16T14:54:56.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.959+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5388003a00 con 0x7f53980b4240 2024-01-16T14:54:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.960+0000 7f5394ff9700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 0x7f537806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.960+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5388088fd0 con 0x7f53980b4240 2024-01-16T14:54:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.960+0000 7f538ffff700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 0x7f537806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:56.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.960+0000 7f538ffff700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 0x7f537806d810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5380009fd0 tx=0x7f538001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:56.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:56.964+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5388053ed0 con 0x7f53980b4240 2024-01-16T14:54:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.174+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53980c5680 con 0x7f53980b4240 2024-01-16T14:54:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.175+0000 7f5394ff9700 1 -- 172.21.15.26:0/753624684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5388003cb0 con 0x7f53980b4240 2024-01-16T14:54:57.158 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:54:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 msgr2=0x7f537806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 0x7f537806d810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5380009fd0 tx=0x7f538001c040 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 msgr2=0x7f53981f6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f538800c520 tx=0x7f538800c830 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 shutdown_connections 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53980c4950 0x7f539815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f537806b350 0x7f537806d810 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53980b4240 0x7f53981f6990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 --2- 172.21.15.26:0/753624684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539815bde0 0x7f53980c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 >> 172.21.15.26:0/753624684 conn(0x7f53980b3720 msgr2=0x7f53980cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 shutdown_connections 2024-01-16T14:54:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.177+0000 7f539dbf1700 1 -- 172.21.15.26:0/753624684 wait complete. 2024-01-16T14:54:57.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:54:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.331+0000 7f247c8d9700 1 Processor -- start 2024-01-16T14:54:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.331+0000 7f247c8d9700 1 -- start start 2024-01-16T14:54:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f247c8d9700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24781440f0 0x7f24780b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f247c8d9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2478143ec0 con 0x7f24780722c0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f247c8d9700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24780b3ae0 con 0x7f2478143ae0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f247c8d9700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780b40a0 con 0x7f24781440f0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2476d9d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24781440f0 0x7f24780b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2475d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24780722c0 0x7f247806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58612/0 (socket says 172.21.15.26:58612) 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2475d9b700 1 -- 172.21.15.26:0/2430641892 learned_addr learned my addr 172.21.15.26:0/2430641892 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2476d9d700 1 -- 172.21.15.26:0/2430641892 >> v1:172.21.15.93:6789/0 conn(0x7f2478143ae0 legacy=0x7f2478071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2476d9d700 1 -- 172.21.15.26:0/2430641892 >> v1:172.21.15.26:6789/0 conn(0x7f24780722c0 legacy=0x7f247806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2476d9d700 1 -- 172.21.15.26:0/2430641892 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24780b3dc0 con 0x7f24781440f0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.332+0000 7f2476d9d700 1 --2- 172.21.15.26:0/2430641892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24781440f0 0x7f24780b51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2468009840 tx=0x7f2468009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d16c437cefe2226 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247559a700 1 -- 172.21.15.26:0/2430641892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468011830 con 0x7f24781440f0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247559a700 1 -- 172.21.15.26:0/2430641892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2468011990 con 0x7f24781440f0 2024-01-16T14:54:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247559a700 1 -- 172.21.15.26:0/2430641892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468011d20 con 0x7f24781440f0 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 -- 172.21.15.26:0/2430641892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24781440f0 msgr2=0x7f24780b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 --2- 172.21.15.26:0/2430641892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24781440f0 0x7f24780b51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2468009840 tx=0x7f2468009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 -- 172.21.15.26:0/2430641892 shutdown_connections 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 --2- 172.21.15.26:0/2430641892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24781440f0 0x7f24780b51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 -- 172.21.15.26:0/2430641892 >> 172.21.15.26:0/2430641892 conn(0x7f24780b2740 msgr2=0x7f24780b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 -- 172.21.15.26:0/2430641892 shutdown_connections 2024-01-16T14:54:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.333+0000 7f247c8d9700 1 -- 172.21.15.26:0/2430641892 wait complete. 2024-01-16T14:54:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.335+0000 7f247c8d9700 1 Processor -- start 2024-01-16T14:54:57.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.335+0000 7f247c8d9700 1 -- start start 2024-01-16T14:54:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2478143ae0 0x7f24781edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24781440f0 0x7f2478142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781f4de0 con 0x7f24780722c0 2024-01-16T14:54:57.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478142e30 con 0x7f24781440f0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478143110 con 0x7f2478143ae0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55418/0 (socket says 172.21.15.26:55418) 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 -- 172.21.15.26:0/932425623 learned_addr learned my addr 172.21.15.26:0/932425623 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f2476d9d700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24781440f0 0x7f2478142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f2475d9b700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2478143ae0 0x7f24781edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 -- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2478143ae0 msgr2=0x7f24781edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2478143ae0 0x7f24781edd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 -- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24781440f0 msgr2=0x7f2478142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24781440f0 0x7f2478142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 -- 172.21.15.26:0/932425623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680094a0 con 0x7f24780722c0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247659c700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2470001ed0 tx=0x7f247000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000ade0 con 0x7f24780722c0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24700020d0 con 0x7f24780722c0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24781f40a0 con 0x7f24780722c0 2024-01-16T14:54:57.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.336+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470003c20 con 0x7f24780722c0 2024-01-16T14:54:57.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.337+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781ee260 con 0x7f24780722c0 2024-01-16T14:54:57.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.337+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24781f45c0 con 0x7f24780722c0 2024-01-16T14:54:57.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.338+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f24700050d0 con 0x7f24780722c0 2024-01-16T14:54:57.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.338+0000 7f246effd700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 0x7f245806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.338+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2470089ca0 con 0x7f24780722c0 2024-01-16T14:54:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.339+0000 7f2475d9b700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 0x7f245806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.339+0000 7f2475d9b700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 0x7f245806d810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2478140250 tx=0x7f2460009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.343+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24781f45c0 con 0x7f24780722c0 2024-01-16T14:54:57.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.553+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24781f45c0 con 0x7f24780722c0 2024-01-16T14:54:57.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.553+0000 7f246effd700 1 -- 172.21.15.26:0/932425623 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2470054ba0 con 0x7f24780722c0 2024-01-16T14:54:57.537 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 msgr2=0x7f245806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 0x7f245806d810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2478140250 tx=0x7f2460009040 comp rx=0 tx=0).stop 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 msgr2=0x7f24781f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2470001ed0 tx=0x7f247000a340 comp rx=0 tx=0).stop 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 shutdown_connections 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24781440f0 0x7f2478142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f245806b350 0x7f245806d810 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24780722c0 0x7f24781f48a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 --2- 172.21.15.26:0/932425623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2478143ae0 0x7f24781edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 >> 172.21.15.26:0/932425623 conn(0x7f24780b2740 msgr2=0x7f247806efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 shutdown_connections 2024-01-16T14:54:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.556+0000 7f247c8d9700 1 -- 172.21.15.26:0/932425623 wait complete. 2024-01-16T14:54:57.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:54:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.706+0000 7f3f27f1f700 1 Processor -- start 2024-01-16T14:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 -- start start 2024-01-16T14:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3f200722c0 0x7f3f200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f20143ec0 con 0x7f3f20143ae0 2024-01-16T14:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f200b40a0 con 0x7f3f200722c0 2024-01-16T14:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.707+0000 7f3f27f1f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200b3ae0 con 0x7f3f201440f0 2024-01-16T14:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f264bc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3f200722c0 0x7f3f200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3f200722c0 0x7f3f200b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:37162/0 (socket says 172.21.15.26:37162) 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f264bc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:59428/0 (socket says 172.21.15.26:59428) 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 -- 172.21.15.26:0/2525352384 learned_addr learned my addr 172.21.15.26:0/2525352384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f25cbb700 1 --1- 172.21.15.26:0/2525352384 >> v1:172.21.15.93:6789/0 conn(0x7f3f20143ae0 0x7f3f20071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:60442/0 (socket says 172.21.15.26:60442) 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 -- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 msgr2=0x7f3f2006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 -- 172.21.15.26:0/2525352384 >> v1:172.21.15.93:6789/0 conn(0x7f3f20143ae0 legacy=0x7f3f20071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 -- 172.21.15.26:0/2525352384 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200b3dc0 con 0x7f3f200722c0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f264bc700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.708+0000 7f3f254ba700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f200b51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00bb90 tx=0x7f3f1c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35fc898ad5629ddb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f24cb9700 1 -- 172.21.15.26:0/2525352384 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c00dd30 con 0x7f3f200722c0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f24cb9700 1 -- 172.21.15.26:0/2525352384 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c00de90 con 0x7f3f200722c0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f24cb9700 1 -- 172.21.15.26:0/2525352384 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c010690 con 0x7f3f200722c0 2024-01-16T14:54:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 -- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 msgr2=0x7f3f200b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f200b51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c00bb90 tx=0x7f3f1c00bea0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 -- 172.21.15.26:0/2525352384 shutdown_connections 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/2525352384 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f200b51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 -- 172.21.15.26:0/2525352384 >> 172.21.15.26:0/2525352384 conn(0x7f3f200b2740 msgr2=0x7f3f200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 -- 172.21.15.26:0/2525352384 shutdown_connections 2024-01-16T14:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.709+0000 7f3f27f1f700 1 -- 172.21.15.26:0/2525352384 wait complete. 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 Processor -- start 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 -- start start 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f20137330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f201510d0 con 0x7f3f20143ae0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20150450 con 0x7f3f200722c0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.711+0000 7f3f27f1f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20150730 con 0x7f3f201440f0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55420/0 (socket says 172.21.15.26:55420) 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 -- 172.21.15.26:0/503978637 learned_addr learned my addr 172.21.15.26:0/503978637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f264bc700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f25cbb700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f20137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 -- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 msgr2=0x7f3f2015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 -- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 msgr2=0x7f3f20137330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f20137330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 -- 172.21.15.26:0/503978637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f18009a70 con 0x7f3f20143ae0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f254ba700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c000c00 tx=0x7f3f1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c01f070 con 0x7f3f20143ae0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c0038b0 con 0x7f3f20143ae0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c003c60 con 0x7f3f20143ae0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c00b7f0 con 0x7f3f20143ae0 2024-01-16T14:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.712+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f20150cf0 con 0x7f3f20143ae0 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.713+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f20151210 con 0x7f3f20143ae0 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.714+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3f1c019400 con 0x7f3f20143ae0 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.715+0000 7f3f127fc700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 0x7f3f0406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.715+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f1c08cd00 con 0x7f3f20143ae0 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.715+0000 7f3f25cbb700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 0x7f3f0406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.715+0000 7f3f25cbb700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 0x7f3f0406d810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.719+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f1c057ce0 con 0x7f3f20143ae0 2024-01-16T14:54:57.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.980+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f20151210 con 0x7f3f20143ae0 2024-01-16T14:54:57.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.981+0000 7f3f127fc700 1 -- 172.21.15.26:0/503978637 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f3f20151210 con 0x7f3f20143ae0 2024-01-16T14:54:57.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:57.964 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 msgr2=0x7f3f0406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 0x7f3f0406d810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f60 tx=0x7f3f14007ed0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 msgr2=0x7f3f2015c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c000c00 tx=0x7f3f1c004750 comp rx=0 tx=0).stop 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 shutdown_connections 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3f201440f0 0x7f3f2015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3f200722c0 0x7f3f20137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3f0406b350 0x7f3f0406d810 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 --2- 172.21.15.26:0/503978637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f20143ae0 0x7f3f2015c6c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:57.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 >> 172.21.15.26:0/503978637 conn(0x7f3f200b2740 msgr2=0x7f3f200b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:57.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 shutdown_connections 2024-01-16T14:54:57.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:57.984+0000 7f3f27f1f700 1 -- 172.21.15.26:0/503978637 wait complete. 2024-01-16T14:54:57.968 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:54:57.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:54:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.133+0000 7fc38f166700 1 Processor -- start 2024-01-16T14:54:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.133+0000 7fc38f166700 1 -- start start 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.134+0000 7fc38f166700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc38815be40 0x7fc3880af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.134+0000 7fc38f166700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3880d0ec0 0x7fc3880b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc38f166700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3880af940 con 0x7fc3880b1a90 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc38f166700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3880b1130 con 0x7fc3880d0ec0 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc38f166700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc388067420 con 0x7fc38815be40 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3880d0ec0 0x7fc3880b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3880d0ec0 0x7fc3880b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55428/0 (socket says 172.21.15.26:55428) 2024-01-16T14:54:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 -- 172.21.15.26:0/139106336 learned_addr learned my addr 172.21.15.26:0/139106336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc38cf02700 1 --2- 172.21.15.26:0/139106336 >> v2:172.21.15.93:3300/0 conn(0x7fc38815be40 0x7fc3880af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 -- 172.21.15.26:0/139106336 >> v2:172.21.15.93:3300/0 conn(0x7fc38815be40 msgr2=0x7fc3880af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 --2- 172.21.15.26:0/139106336 >> v2:172.21.15.93:3300/0 conn(0x7fc38815be40 0x7fc3880af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 -- 172.21.15.26:0/139106336 >> v1:172.21.15.93:6789/0 conn(0x7fc3880b1a90 legacy=0x7fc38806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 -- 172.21.15.26:0/139106336 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3880b0ab0 con 0x7fc3880d0ec0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc387fff700 1 --2- 172.21.15.26:0/139106336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880b1550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3880737e0 tx=0x7fc370009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305bebde04457d13 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc3877fe700 1 -- 172.21.15.26:0/139106336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370011830 con 0x7fc3880d0ec0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc3877fe700 1 -- 172.21.15.26:0/139106336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc370011990 con 0x7fc3880d0ec0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.135+0000 7fc3877fe700 1 -- 172.21.15.26:0/139106336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370011d20 con 0x7fc3880d0ec0 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 -- 172.21.15.26:0/139106336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 msgr2=0x7fc3880b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 --2- 172.21.15.26:0/139106336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880b1550 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3880737e0 tx=0x7fc370009bd0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 -- 172.21.15.26:0/139106336 shutdown_connections 2024-01-16T14:54:58.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 --2- 172.21.15.26:0/139106336 >> v2:172.21.15.93:3300/0 conn(0x7fc38815be40 0x7fc3880af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 --2- 172.21.15.26:0/139106336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880b1550 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 -- 172.21.15.26:0/139106336 >> 172.21.15.26:0/139106336 conn(0x7fc3880717f0 msgr2=0x7fc3880b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 -- 172.21.15.26:0/139106336 shutdown_connections 2024-01-16T14:54:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.136+0000 7fc38f166700 1 -- 172.21.15.26:0/139106336 wait complete. 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.137+0000 7fc38f166700 1 Processor -- start 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.137+0000 7fc38f166700 1 -- start start 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 0x7fc3881f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3881546b0 0x7fc3880cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880ad730 con 0x7fc3880d0ec0 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880cb950 con 0x7fc38815be40 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38f166700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc388156180 con 0x7fc3881546b0 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38cf02700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38cf02700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55430/0 (socket says 172.21.15.26:55430) 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.138+0000 7fc38cf02700 1 -- 172.21.15.26:0/69154678 learned_addr learned my addr 172.21.15.26:0/69154678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38d703700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3881546b0 0x7fc3880cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc387fff700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 0x7fc3881f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 -- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3881546b0 msgr2=0x7fc3880cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3881546b0 0x7fc3880cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 -- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 msgr2=0x7fc3881f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 0x7fc3881f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 -- 172.21.15.26:0/69154678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370009550 con 0x7fc3880d0ec0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc387fff700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 0x7fc3881f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38cf02700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc37800e870 tx=0x7fc37800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378009590 con 0x7fc3880d0ec0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37800ee70 con 0x7fc3880d0ec0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880cbbf0 con 0x7fc3880d0ec0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800ca50 con 0x7fc3880d0ec0 2024-01-16T14:54:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.139+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc388157eb0 con 0x7fc3880d0ec0 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.140+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc388075480 con 0x7fc3880d0ec0 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.141+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc37800cbb0 con 0x7fc3880d0ec0 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.141+0000 7fc3857fa700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 0x7fc36806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.141+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc378089b10 con 0x7fc3880d0ec0 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.141+0000 7fc387fff700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 0x7fc36806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.142+0000 7fc387fff700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 0x7fc36806d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ccf10 tx=0x7fc370008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.146+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc378054980 con 0x7fc3880d0ec0 2024-01-16T14:54:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.359+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3880b1450 con 0x7fc3880d0ec0 2024-01-16T14:54:58.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.360+0000 7fc3857fa700 1 -- 172.21.15.26:0/69154678 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fc378021090 con 0x7fc3880d0ec0 2024-01-16T14:54:58.344 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.362+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 msgr2=0x7fc36806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 0x7fc36806d810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3880ccf10 tx=0x7fc370008030 comp rx=0 tx=0).stop 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 msgr2=0x7fc3880ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc37800e870 tx=0x7fc37800ec30 comp rx=0 tx=0).stop 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 shutdown_connections 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3881546b0 0x7fc3880cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc36806b350 0x7fc36806d810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3880d0ec0 0x7fc3880ad1f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 --2- 172.21.15.26:0/69154678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc38815be40 0x7fc3881f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 >> 172.21.15.26:0/69154678 conn(0x7fc3880717f0 msgr2=0x7fc3880b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 shutdown_connections 2024-01-16T14:54:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.363+0000 7fc38f166700 1 -- 172.21.15.26:0/69154678 wait complete. 2024-01-16T14:54:58.360 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:54:58.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:54:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.518+0000 7f38e24df700 1 Processor -- start 2024-01-16T14:54:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.518+0000 7f38e24df700 1 -- start start 2024-01-16T14:54:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38e24df700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38dc1478c0 0x7f38dc0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38e24df700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38e24df700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc06dbe0 con 0x7f38dc06b510 2024-01-16T14:54:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38e24df700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc06b9a0 con 0x7f38dc1478c0 2024-01-16T14:54:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38e24df700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc147ca0 con 0x7f38dc161a90 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38dbfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38dc1478c0 0x7f38dc0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38dbfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38dc1478c0 0x7f38dc0b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55436/0 (socket says 172.21.15.26:55436) 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38dbfff700 1 -- 172.21.15.26:0/1861757083 learned_addr learned my addr 172.21.15.26:0/1861757083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.519+0000 7f38db7fe700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38dbfff700 1 -- 172.21.15.26:0/1861757083 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 msgr2=0x7f38dc06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38dbfff700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc06afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38daffd700 1 -- 172.21.15.26:0/1861757083 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929611562 0 0) 0x7f38dc06dbe0 con 0x7f38dc06b510 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38dbfff700 1 -- 172.21.15.26:0/1861757083 >> v1:172.21.15.93:6789/0 conn(0x7f38dc06b510 legacy=0x7f38dc071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38dbfff700 1 -- 172.21.15.26:0/1861757083 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc1622e0 con 0x7f38dc1478c0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38dbfff700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc0b57b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0098f0 tx=0x7f38cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83528fe735be5cd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38daffd700 1 -- 172.21.15.26:0/1861757083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011830 con 0x7f38dc1478c0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38daffd700 1 -- 172.21.15.26:0/1861757083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc011990 con 0x7f38dc1478c0 2024-01-16T14:54:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.520+0000 7f38daffd700 1 -- 172.21.15.26:0/1861757083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc011d20 con 0x7f38dc1478c0 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 -- 172.21.15.26:0/1861757083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 msgr2=0x7f38dc0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc0b57b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0098f0 tx=0x7f38cc009c00 comp rx=0 tx=0).stop 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 -- 172.21.15.26:0/1861757083 shutdown_connections 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc0b57b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 --2- 172.21.15.26:0/1861757083 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc06afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 -- 172.21.15.26:0/1861757083 >> 172.21.15.26:0/1861757083 conn(0x7f38dc0d0aa0 msgr2=0x7f38dc0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 -- 172.21.15.26:0/1861757083 shutdown_connections 2024-01-16T14:54:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.521+0000 7f38e24df700 1 -- 172.21.15.26:0/1861757083 wait complete. 2024-01-16T14:54:58.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.522+0000 7f38e24df700 1 Processor -- start 2024-01-16T14:54:58.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.522+0000 7f38e24df700 1 -- start start 2024-01-16T14:54:58.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc06b510 0x7f38dc0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0b1a20 con 0x7f38dc1478c0 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0b1460 con 0x7f38dc06b510 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e24df700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0b1740 con 0x7f38dc161a90 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55448/0 (socket says 172.21.15.26:55448) 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38db7fe700 1 -- 172.21.15.26:0/3755202274 learned_addr learned my addr 172.21.15.26:0/3755202274 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38dbfff700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc06b510 0x7f38dc0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.523+0000 7f38e0a7c700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 -- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 msgr2=0x7f38dc0ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc0ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 -- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc06b510 msgr2=0x7f38dc0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc06b510 0x7f38dc0b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 -- 172.21.15.26:0/3755202274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009550 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38db7fe700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e870 tx=0x7f38d000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0009590 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0013680 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00139f0 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc139400 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.524+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc0af2e0 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.525+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc139a50 con 0x7f38dc1478c0 2024-01-16T14:54:58.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.526+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f38d0013b50 con 0x7f38dc1478c0 2024-01-16T14:54:58.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.526+0000 7f38d8ff9700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 0x7f38c806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.526+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38d0089980 con 0x7f38dc1478c0 2024-01-16T14:54:58.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.526+0000 7f38dbfff700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 0x7f38c806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.527+0000 7f38dbfff700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 0x7f38c806d810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c40 tx=0x7f38cc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.531+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38d0054880 con 0x7f38dc1478c0 2024-01-16T14:54:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.738+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38dc139260 con 0x7f38dc1478c0 2024-01-16T14:54:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.738+0000 7f38d8ff9700 1 -- 172.21.15.26:0/3755202274 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38dc139260 con 0x7f38dc1478c0 2024-01-16T14:54:58.721 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 msgr2=0x7f38c806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 0x7f38c806d810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009c40 tx=0x7f38cc008030 comp rx=0 tx=0).stop 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 msgr2=0x7f38dc1f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38d000e870 tx=0x7f38d000eb80 comp rx=0 tx=0).stop 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 shutdown_connections 2024-01-16T14:54:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38dc161a90 0x7f38dc0ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38dc06b510 0x7f38dc0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38c806b350 0x7f38c806d810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 --2- 172.21.15.26:0/3755202274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38dc1478c0 0x7f38dc1f6a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 >> 172.21.15.26:0/3755202274 conn(0x7f38dc0d0aa0 msgr2=0x7f38dc1453e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 shutdown_connections 2024-01-16T14:54:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.741+0000 7f38e24df700 1 -- 172.21.15.26:0/3755202274 wait complete. 2024-01-16T14:54:58.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:54:58.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.894+0000 7f28bcb37700 1 Processor -- start 2024-01-16T14:54:58.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.894+0000 7f28bcb37700 1 -- start start 2024-01-16T14:54:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.895+0000 7f28bcb37700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28b815be40 0x7f28b80af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.895+0000 7f28bcb37700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b80af940 con 0x7f28b80d0ec0 2024-01-16T14:54:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.895+0000 7f28bcb37700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b80b0ab0 con 0x7f28b806b7c0 2024-01-16T14:54:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.895+0000 7f28bcb37700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28b80b1130 con 0x7f28b815be40 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b659c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28b815be40 0x7f28b80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b5d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28b80d0ec0 0x7f28b80b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b5d9b700 1 -- 172.21.15.26:0/303973718 learned_addr learned my addr 172.21.15.26:0/303973718 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b659c700 1 -- 172.21.15.26:0/303973718 >> v1:172.21.15.93:6789/0 conn(0x7f28b806b7c0 legacy=0x7f28b815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b659c700 1 -- 172.21.15.26:0/303973718 >> v1:172.21.15.26:6789/0 conn(0x7f28b80d0ec0 legacy=0x7f28b80b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b559a700 1 -- 172.21.15.26:0/303973718 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353941563 0 0) 0x7f28ac001110 con 0x7f28b80d0ec0 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b659c700 1 -- 172.21.15.26:0/303973718 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80b1c60 con 0x7f28b815be40 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b659c700 1 --2- 172.21.15.26:0/303973718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b815be40 0x7f28b80af400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009840 tx=0x7f28a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09ffe7f2d7df432 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b559a700 1 -- 172.21.15.26:0/303973718 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8011830 con 0x7f28b815be40 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b559a700 1 -- 172.21.15.26:0/303973718 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a8011990 con 0x7f28b815be40 2024-01-16T14:54:58.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.896+0000 7f28b559a700 1 -- 172.21.15.26:0/303973718 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8011d20 con 0x7f28b815be40 2024-01-16T14:54:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 -- 172.21.15.26:0/303973718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b815be40 msgr2=0x7f28b80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 --2- 172.21.15.26:0/303973718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b815be40 0x7f28b80af400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009840 tx=0x7f28a8009b50 comp rx=0 tx=0).stop 2024-01-16T14:54:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 -- 172.21.15.26:0/303973718 shutdown_connections 2024-01-16T14:54:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 --2- 172.21.15.26:0/303973718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b815be40 0x7f28b80af400 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 -- 172.21.15.26:0/303973718 >> 172.21.15.26:0/303973718 conn(0x7f28b80717f0 msgr2=0x7f28b80b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:58.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 -- 172.21.15.26:0/303973718 shutdown_connections 2024-01-16T14:54:58.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.897+0000 7f28bcb37700 1 -- 172.21.15.26:0/303973718 wait complete. 2024-01-16T14:54:58.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.898+0000 7f28bcb37700 1 Processor -- start 2024-01-16T14:54:58.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 -- start start 2024-01-16T14:54:58.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28b80d0ec0 0x7f28b8149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28b815be40 0x7f28b809ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b806deb0 con 0x7f28b806b7c0 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b814a230 con 0x7f28b815be40 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28bcb37700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b806d8f0 con 0x7f28b80d0ec0 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28b659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28b659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55466/0 (socket says 172.21.15.26:55466) 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.899+0000 7f28b659c700 1 -- 172.21.15.26:0/2851352075 learned_addr learned my addr 172.21.15.26:0/2851352075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b5d9b700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28b80d0ec0 0x7f28b8149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b6d9d700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28b815be40 0x7f28b809ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 -- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28b80d0ec0 msgr2=0x7f28b8149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28b80d0ec0 0x7f28b8149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 -- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28b815be40 msgr2=0x7f28b809ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28b815be40 0x7f28b809ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 -- 172.21.15.26:0/2851352075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a80094a0 con 0x7f28b806b7c0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28b659c700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009b90 tx=0x7f28a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a8004590 con 0x7f28b806b7c0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a801e070 con 0x7f28b806b7c0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b806dbd0 con 0x7f28b806b7c0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a80038c0 con 0x7f28b806b7c0 2024-01-16T14:54:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.900+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b814a7b0 con 0x7f28b806b7c0 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.901+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b8075480 con 0x7f28b806b7c0 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.902+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f28a8003a20 con 0x7f28b806b7c0 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.902+0000 7f28a6ffd700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 0x7f289806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.902+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28a808ab50 con 0x7f28b806b7c0 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.902+0000 7f28b5d9b700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 0x7f289806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:58.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.903+0000 7f28b5d9b700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 0x7f289806d810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28b80d0cb0 tx=0x7f28ac007e10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:58.907+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28a8055a50 con 0x7f28b806b7c0 2024-01-16T14:54:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.114+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28b80af4a0 con 0x7f28b806b7c0 2024-01-16T14:54:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.115+0000 7f28a6ffd700 1 -- 172.21.15.26:0/2851352075 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28a8015020 con 0x7f28b806b7c0 2024-01-16T14:54:59.098 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:54:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 msgr2=0x7f289806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 0x7f289806d810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28b80d0cb0 tx=0x7f28ac007e10 comp rx=0 tx=0).stop 2024-01-16T14:54:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 msgr2=0x7f28b809a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009b90 tx=0x7f28a8004750 comp rx=0 tx=0).stop 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 shutdown_connections 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28b815be40 0x7f28b809ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f289806b350 0x7f289806d810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b806b7c0 0x7f28b809a370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 --2- 172.21.15.26:0/2851352075 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28b80d0ec0 0x7f28b8149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 >> 172.21.15.26:0/2851352075 conn(0x7f28b80717f0 msgr2=0x7f28b80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 shutdown_connections 2024-01-16T14:54:59.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.117+0000 7f28bcb37700 1 -- 172.21.15.26:0/2851352075 wait complete. 2024-01-16T14:54:59.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:54:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.270+0000 7fd277d5a700 1 Processor -- start 2024-01-16T14:54:59.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.270+0000 7fd277d5a700 1 -- start start 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd277d5a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd270158730 0x7fd27006f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd277d5a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 0x7fd2700b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd277d5a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27015e990 con 0x7fd27006f9e0 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd277d5a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27015ec70 con 0x7fd270158730 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd277d5a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27015ef50 con 0x7fd2700722c0 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd2752f5700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd27006f9e0 0x7fd270071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58632/0 (socket says 172.21.15.26:58632) 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd2752f5700 1 -- 172.21.15.26:0/1896140498 learned_addr learned my addr 172.21.15.26:0/1896140498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.271+0000 7fd2762f7700 1 --2- 172.21.15.26:0/1896140498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 0x7fd2700b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd275af6700 1 --2- 172.21.15.26:0/1896140498 >> v2:172.21.15.93:3300/0 conn(0x7fd270158730 0x7fd27006f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114276315 0 0) 0x7fd27015e990 con 0x7fd27006f9e0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd25c0035b0 con 0x7fd27006f9e0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd2762f7700 1 -- 172.21.15.26:0/1896140498 >> v2:172.21.15.93:3300/0 conn(0x7fd270158730 msgr2=0x7fd27006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd2762f7700 1 --2- 172.21.15.26:0/1896140498 >> v2:172.21.15.93:3300/0 conn(0x7fd270158730 0x7fd27006f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 604725262 0 0) 0x7fd25c0035b0 con 0x7fd27006f9e0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd2762f7700 1 -- 172.21.15.26:0/1896140498 >> v1:172.21.15.26:6789/0 conn(0x7fd27006f9e0 legacy=0x7fd270071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd2762f7700 1 -- 172.21.15.26:0/1896140498 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27015f230 con 0x7fd2700722c0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd2762f7700 1 --2- 172.21.15.26:0/1896140498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 0x7fd2700b3c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd26800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38397073252658dc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26800e070 con 0x7fd2700722c0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd268003a20 con 0x7fd2700722c0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.272+0000 7fd274af4700 1 -- 172.21.15.26:0/1896140498 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260001670 con 0x7fd2700722c0 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 -- 172.21.15.26:0/1896140498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 msgr2=0x7fd2700b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 --2- 172.21.15.26:0/1896140498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 0x7fd2700b3c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd268009d90 tx=0x7fd26800b330 comp rx=0 tx=0).stop 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 -- 172.21.15.26:0/1896140498 shutdown_connections 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 --2- 172.21.15.26:0/1896140498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2700722c0 0x7fd2700b3c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 --2- 172.21.15.26:0/1896140498 >> v2:172.21.15.93:3300/0 conn(0x7fd270158730 0x7fd27006f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 -- 172.21.15.26:0/1896140498 >> 172.21.15.26:0/1896140498 conn(0x7fd2700b27b0 msgr2=0x7fd2700b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 -- 172.21.15.26:0/1896140498 shutdown_connections 2024-01-16T14:54:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.273+0000 7fd277d5a700 1 -- 172.21.15.26:0/1896140498 wait complete. 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.274+0000 7fd277d5a700 1 Processor -- start 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.274+0000 7fd277d5a700 1 -- start start 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2700722c0 0x7fd2701eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd270158730 0x7fd2701527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd270155010 con 0x7fd27006f9e0 2024-01-16T14:54:59.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd270150ed0 con 0x7fd2700722c0 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd277d5a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2701f1e50 con 0x7fd270158730 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55474/0 (socket says 172.21.15.26:55474) 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 -- 172.21.15.26:0/2400656643 learned_addr learned my addr 172.21.15.26:0/2400656643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd2762f7700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd270158730 0x7fd2701527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd2752f5700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2700722c0 0x7fd2701eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 -- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd270158730 msgr2=0x7fd2701527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd270158730 0x7fd2701527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 -- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2700722c0 msgr2=0x7fd2701eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2700722c0 0x7fd2701eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.275+0000 7fd275af6700 1 -- 172.21.15.26:0/2400656643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009a70 con 0x7fd27006f9e0 2024-01-16T14:54:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd275af6700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e870 tx=0x7fd26c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c009590 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c00ee70 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2701f2130 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00ca50 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.276+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2701f26f0 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.277+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd270075480 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.277+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd26c00cbb0 con 0x7fd27006f9e0 2024-01-16T14:54:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.278+0000 7fd2667fc700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 0x7fd25406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.278+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd26c089b40 con 0x7fd27006f9e0 2024-01-16T14:54:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.278+0000 7fd2752f5700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 0x7fd25406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.279+0000 7fd2752f5700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 0x7fd25406d810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd260001eb0 tx=0x7fd260001ce0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.283+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd26c0549c0 con 0x7fd27006f9e0 2024-01-16T14:54:59.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.498+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2700cfdf0 con 0x7fd27006f9e0 2024-01-16T14:54:59.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.498+0000 7fd2667fc700 1 -- 172.21.15.26:0/2400656643 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd26c00ce60 con 0x7fd27006f9e0 2024-01-16T14:54:59.481 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:54:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.500+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 msgr2=0x7fd25406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.500+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 0x7fd25406d810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd260001eb0 tx=0x7fd260001ce0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.500+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 msgr2=0x7fd270154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.500+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e870 tx=0x7fd26c00ec30 comp rx=0 tx=0).stop 2024-01-16T14:54:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 shutdown_connections 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd270158730 0x7fd2701527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd25406b350 0x7fd25406d810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27006f9e0 0x7fd270154ad0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 --2- 172.21.15.26:0/2400656643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2700722c0 0x7fd2701eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 >> 172.21.15.26:0/2400656643 conn(0x7fd2700b27b0 msgr2=0x7fd2700b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 shutdown_connections 2024-01-16T14:54:59.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.501+0000 7fd277d5a700 1 -- 172.21.15.26:0/2400656643 wait complete. 2024-01-16T14:54:59.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:54:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.651+0000 7f0cfe164700 1 Processor -- start 2024-01-16T14:54:59.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.651+0000 7f0cfe164700 1 -- start start 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cfe164700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cf80726a0 0x7f0cf8072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cfe164700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf815f3e0 con 0x7f0cf8072fc0 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cfe164700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf806c8e0 con 0x7f0cf80b3d90 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cfe164700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf80affd0 con 0x7f0cf80726a0 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cf80726a0 0x7f0cf8072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf7fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0cf8072fc0 0x7f0cf806c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58634/0 (socket says 172.21.15.26:58634) 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf7fff700 1 -- 172.21.15.26:0/3489108412 learned_addr learned my addr 172.21.15.26:0/3489108412 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf77fe700 1 -- 172.21.15.26:0/3489108412 >> v1:172.21.15.93:6789/0 conn(0x7f0cf80b3d90 legacy=0x7f0cf806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf77fe700 1 -- 172.21.15.26:0/3489108412 >> v1:172.21.15.26:6789/0 conn(0x7f0cf8072fc0 legacy=0x7f0cf806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf77fe700 1 -- 172.21.15.26:0/3489108412 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf815e7d0 con 0x7f0cf80726a0 2024-01-16T14:54:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.652+0000 7f0cf67fc700 1 -- 172.21.15.26:0/3489108412 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185631896 0 0) 0x7f0cf815f3e0 con 0x7f0cf8072fc0 2024-01-16T14:54:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cf77fe700 1 --2- 172.21.15.26:0/3489108412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80726a0 0x7f0cf8072a80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd815b089cb84a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cf67fc700 1 -- 172.21.15.26:0/3489108412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8011830 con 0x7f0cf80726a0 2024-01-16T14:54:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cf67fc700 1 -- 172.21.15.26:0/3489108412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce8011990 con 0x7f0cf80726a0 2024-01-16T14:54:59.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cf67fc700 1 -- 172.21.15.26:0/3489108412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8011d20 con 0x7f0cf80726a0 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 -- 172.21.15.26:0/3489108412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80726a0 msgr2=0x7f0cf8072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 --2- 172.21.15.26:0/3489108412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80726a0 0x7f0cf8072a80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce8009b80 comp rx=0 tx=0).stop 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 -- 172.21.15.26:0/3489108412 shutdown_connections 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 --2- 172.21.15.26:0/3489108412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80726a0 0x7f0cf8072a80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 -- 172.21.15.26:0/3489108412 >> 172.21.15.26:0/3489108412 conn(0x7f0cf80717f0 msgr2=0x7f0cf80b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 -- 172.21.15.26:0/3489108412 shutdown_connections 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.653+0000 7f0cfe164700 1 -- 172.21.15.26:0/3489108412 wait complete. 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.655+0000 7f0cfe164700 1 Processor -- start 2024-01-16T14:54:59.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.655+0000 7f0cfe164700 1 -- start start 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cf80726a0 0x7f0cf81482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cf8072fc0 0x7f0cf8145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8146af0 con 0x7f0cf80b3d90 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8145bd0 con 0x7f0cf80726a0 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cfe164700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8145e70 con 0x7f0cf8072fc0 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55484/0 (socket says 172.21.15.26:55484) 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 -- 172.21.15.26:0/1718501145 learned_addr learned my addr 172.21.15.26:0/1718501145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf77fe700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cf80726a0 0x7f0cf81482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf6ffd700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cf8072fc0 0x7f0cf8145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 -- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cf8072fc0 msgr2=0x7f0cf8145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cf8072fc0 0x7f0cf8145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 -- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cf80726a0 msgr2=0x7f0cf81482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cf80726a0 0x7f0cf81482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 -- 172.21.15.26:0/1718501145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80094a0 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cf7fff700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0002230 tx=0x7f0cf000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.656+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000aac0 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.657+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf000ac40 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.657+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf8146150 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.657+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00070d0 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.657+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf8146710 con 0x7f0cf80b3d90 2024-01-16T14:54:59.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.658+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf8146c30 con 0x7f0cf80b3d90 2024-01-16T14:54:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.658+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0cf0007250 con 0x7f0cf80b3d90 2024-01-16T14:54:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.659+0000 7f0cdffff700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 0x7f0ce406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:54:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.659+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cf0089920 con 0x7f0cf80b3d90 2024-01-16T14:54:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.659+0000 7f0cf77fe700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 0x7f0ce406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:54:59.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.659+0000 7f0cf77fe700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 0x7f0ce406d810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009840 tx=0x7f0ce8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:54:59.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.664+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cf8146c30 con 0x7f0cf80b3d90 2024-01-16T14:54:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.921+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf8146c30 con 0x7f0cf80b3d90 2024-01-16T14:54:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.922+0000 7f0cdffff700 1 -- 172.21.15.26:0/1718501145 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f0cf8146c30 con 0x7f0cf80b3d90 2024-01-16T14:54:59.905 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:54:59.905 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 msgr2=0x7f0ce406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 0x7f0ce406d810 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009840 tx=0x7f0ce8008030 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 msgr2=0x7f0cf8146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0002230 tx=0x7f0cf000cf50 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 shutdown_connections 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0ce406b350 0x7f0ce406d810 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cf80b3d90 0x7f0cf8146f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cf80726a0 0x7f0cf81482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 --2- 172.21.15.26:0/1718501145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cf8072fc0 0x7f0cf8145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 >> 172.21.15.26:0/1718501145 conn(0x7f0cf80717f0 msgr2=0x7f0cf80b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 shutdown_connections 2024-01-16T14:54:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:54:59.924+0000 7f0cfe164700 1 -- 172.21.15.26:0/1718501145 wait complete. 2024-01-16T14:54:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:54:59.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.075+0000 7f5656dda700 1 Processor -- start 2024-01-16T14:55:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.075+0000 7f5656dda700 1 -- start start 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5656dda700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f565015e000 0x7f565006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5656dda700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f565006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5656dda700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500cfd60 con 0x7f56500d0f60 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5656dda700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56500cf4b0 con 0x7f565015e000 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5656dda700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565006f430 con 0x7f56500d0ae0 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5655377700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56500d0f60 0x7f56500cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58642/0 (socket says 172.21.15.26:58642) 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5655377700 1 -- 172.21.15.26:0/2795673264 learned_addr learned my addr 172.21.15.26:0/2795673264 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f5654b76700 1 --2- 172.21.15.26:0/2795673264 >> v2:172.21.15.93:3300/0 conn(0x7f565015e000 0x7f565006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f564ffff700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f565006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352935854 0 0) 0x7f56500cfd60 con 0x7f56500d0f60 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.076+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56400035b0 con 0x7f56500d0f60 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77891368 0 0) 0x7f56400035b0 con 0x7f56500d0f60 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f5654b76700 1 -- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 msgr2=0x7f565006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f5654b76700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f565006f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f5654b76700 1 -- 172.21.15.26:0/2795673264 >> v1:172.21.15.26:6789/0 conn(0x7f56500d0f60 legacy=0x7f56500cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f5654b76700 1 -- 172.21.15.26:0/2795673264 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500cf1d0 con 0x7f565015e000 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f564ffff700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f565006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f5654b76700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565006eef0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5644009840 tx=0x7f5644009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56d7420b413dff4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56480019f0 con 0x7f565015e000 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5644004030 con 0x7f565015e000 2024-01-16T14:55:00.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.077+0000 7f564f7fe700 1 -- 172.21.15.26:0/2795673264 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440043a0 con 0x7f565015e000 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 -- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 msgr2=0x7f565006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565006eef0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5644009840 tx=0x7f5644009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 -- 172.21.15.26:0/2795673264 shutdown_connections 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565006eef0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 --2- 172.21.15.26:0/2795673264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f565006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 -- 172.21.15.26:0/2795673264 >> 172.21.15.26:0/2795673264 conn(0x7f56500b3720 msgr2=0x7f56500b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:00.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 -- 172.21.15.26:0/2795673264 shutdown_connections 2024-01-16T14:55:00.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.078+0000 7f5656dda700 1 -- 172.21.15.26:0/2795673264 wait complete. 2024-01-16T14:55:00.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.079+0000 7f5656dda700 1 Processor -- start 2024-01-16T14:55:00.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.079+0000 7f5656dda700 1 -- start start 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f56501f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500c64c0 con 0x7f56500d0f60 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501f39a0 con 0x7f565015e000 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5656dda700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b4d50 con 0x7f56500d0ae0 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f564ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f564ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55488/0 (socket says 172.21.15.26:55488) 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f564ffff700 1 -- 172.21.15.26:0/182754387 learned_addr learned my addr 172.21.15.26:0/182754387 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5654b76700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f56501f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.080+0000 7f5655377700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 -- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 msgr2=0x7f56501f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f56501f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 -- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 msgr2=0x7f565015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 -- 172.21.15.26:0/182754387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0094a0 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564ffff700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f563c00ec10 tx=0x7f563c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c009760 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c00c850 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56440094a0 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00cbe0 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.081+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56500c5b20 con 0x7f56500d0f60 2024-01-16T14:55:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.082+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5650068050 con 0x7f56500d0f60 2024-01-16T14:55:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.082+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f563c00cd40 con 0x7f56500d0f60 2024-01-16T14:55:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.083+0000 7f564d7fa700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 0x7f563406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.083+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f563c089a50 con 0x7f56500d0f60 2024-01-16T14:55:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.083+0000 7f5654b76700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 0x7f563406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.083+0000 7f5654b76700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 0x7f563406d810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f56440090b0 tx=0x7f5644008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.088+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f563c054950 con 0x7f56500d0f60 2024-01-16T14:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.299+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5650075480 con 0x7f56500d0f60 2024-01-16T14:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.300+0000 7f564d7fa700 1 -- 172.21.15.26:0/182754387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f563c021090 con 0x7f56500d0f60 2024-01-16T14:55:00.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:00.284 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 msgr2=0x7f563406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 0x7f563406d810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f56440090b0 tx=0x7f5644008030 comp rx=0 tx=0).stop 2024-01-16T14:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 msgr2=0x7f56500b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f563c00ec10 tx=0x7f563c00ef20 comp rx=0 tx=0).stop 2024-01-16T14:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 shutdown_connections 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f565015e000 0x7f565015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56500d0ae0 0x7f56501f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f563406b350 0x7f563406d810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 --2- 172.21.15.26:0/182754387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56500d0f60 0x7f56500b4810 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 >> 172.21.15.26:0/182754387 conn(0x7f56500b3720 msgr2=0x7f565006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 shutdown_connections 2024-01-16T14:55:00.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.303+0000 7f5656dda700 1 -- 172.21.15.26:0/182754387 wait complete. 2024-01-16T14:55:00.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.453+0000 7f776032c700 1 Processor -- start 2024-01-16T14:55:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.453+0000 7f776032c700 1 -- start start 2024-01-16T14:55:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f775815bde0 0x7f775806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f775815c1c0 0x7f77580cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77580b4240 0x7f77580cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580cffb0 con 0x7f775815c1c0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580cc670 con 0x7f775815bde0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f776032c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77580cf7a0 con 0x7f77580b4240 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f775815c1c0 0x7f77580cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f775815c1c0 0x7f77580cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55504/0 (socket says 172.21.15.26:55504) 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 -- 172.21.15.26:0/2060813226 learned_addr learned my addr 172.21.15.26:0/2060813226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775e0c8700 1 --2- 172.21.15.26:0/2060813226 >> v2:172.21.15.93:3300/0 conn(0x7f775815bde0 0x7f775806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775e8c9700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77580b4240 0x7f77580cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 -- 172.21.15.26:0/2060813226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77580b4240 msgr2=0x7f77580cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77580b4240 0x7f77580cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 -- 172.21.15.26:0/2060813226 >> v2:172.21.15.93:3300/0 conn(0x7f775815bde0 msgr2=0x7f775806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 --2- 172.21.15.26:0/2060813226 >> v2:172.21.15.93:3300/0 conn(0x7f775815bde0 0x7f775806ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.454+0000 7f775d8c7700 1 -- 172.21.15.26:0/2060813226 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580cdb20 con 0x7f775815c1c0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f775d8c7700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f775815c1c0 0x7f77580cd0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f775806bf70 tx=0x7f7754009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f46c8b3a32c3ce5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f775d0c6700 1 -- 172.21.15.26:0/2060813226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754011830 con 0x7f775815c1c0 2024-01-16T14:55:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f775d0c6700 1 -- 172.21.15.26:0/2060813226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7754011990 con 0x7f775815c1c0 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f775d0c6700 1 -- 172.21.15.26:0/2060813226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754011d20 con 0x7f775815c1c0 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 -- 172.21.15.26:0/2060813226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f775815c1c0 msgr2=0x7f77580cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f775815c1c0 0x7f77580cd0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f775806bf70 tx=0x7f7754009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 -- 172.21.15.26:0/2060813226 shutdown_connections 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f77580b4240 0x7f77580cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 --2- 172.21.15.26:0/2060813226 >> v2:172.21.15.93:3300/0 conn(0x7f775815bde0 0x7f775806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 --2- 172.21.15.26:0/2060813226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f775815c1c0 0x7f77580cd0a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.455+0000 7f776032c700 1 -- 172.21.15.26:0/2060813226 >> 172.21.15.26:0/2060813226 conn(0x7f77580b3720 msgr2=0x7f77580b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.456+0000 7f776032c700 1 -- 172.21.15.26:0/2060813226 shutdown_connections 2024-01-16T14:55:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.456+0000 7f776032c700 1 -- 172.21.15.26:0/2060813226 wait complete. 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.457+0000 7f776032c700 1 Processor -- start 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.457+0000 7f776032c700 1 -- start start 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 0x7f77580c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f775815bde0 0x7f7758140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775815e600 con 0x7f77580c5c40 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775815d580 con 0x7f77580b4240 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f776032c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775815d840 con 0x7f775815bde0 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f775e8c9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f775e8c9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55514/0 (socket says 172.21.15.26:55514) 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.458+0000 7f775e8c9700 1 -- 172.21.15.26:0/3897828706 learned_addr learned my addr 172.21.15.26:0/3897828706 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775d8c7700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f775815bde0 0x7f7758140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e0c8700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 0x7f77580c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 -- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f775815bde0 msgr2=0x7f7758140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f775815bde0 0x7f7758140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 -- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 msgr2=0x7f77580c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 0x7f77580c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 -- 172.21.15.26:0/3897828706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7754009550 con 0x7f77580c5c40 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e0c8700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 0x7f77580c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f775e8c9700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f775000eed0 tx=0x7f775000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750010040 con 0x7f77580c5c40 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000bd30 con 0x7f77580c5c40 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775815db20 con 0x7f77580c5c40 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000bea0 con 0x7f77580c5c40 2024-01-16T14:55:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.459+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775815e0e0 con 0x7f77580c5c40 2024-01-16T14:55:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.460+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775815e300 con 0x7f77580c5c40 2024-01-16T14:55:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.461+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7750014620 con 0x7f77580c5c40 2024-01-16T14:55:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.461+0000 7f774affd700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 0x7f773c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.461+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f775008a050 con 0x7f77580c5c40 2024-01-16T14:55:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.461+0000 7f775e0c8700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 0x7f773c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.462+0000 7f775e0c8700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 0x7f773c06d810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7758152880 tx=0x7f774c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.466+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7750054ec0 con 0x7f77580c5c40 2024-01-16T14:55:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.727+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7758075480 con 0x7f77580c5c40 2024-01-16T14:55:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.728+0000 7f774affd700 1 -- 172.21.15.26:0/3897828706 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f77500584e0 con 0x7f77580c5c40 2024-01-16T14:55:00.711 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:00.711 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:55:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 msgr2=0x7f773c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 0x7f773c06d810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7758152880 tx=0x7f774c008040 comp rx=0 tx=0).stop 2024-01-16T14:55:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 msgr2=0x7f7758141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f775000eed0 tx=0x7f775000b390 comp rx=0 tx=0).stop 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 shutdown_connections 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f773c06b350 0x7f773c06d810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77580c5c40 0x7f7758141280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77580b4240 0x7f77580c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 --2- 172.21.15.26:0/3897828706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f775815bde0 0x7f7758140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 >> 172.21.15.26:0/3897828706 conn(0x7f77580b3720 msgr2=0x7f77580b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 shutdown_connections 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:00.730+0000 7f776032c700 1 -- 172.21.15.26:0/3897828706 wait complete. 2024-01-16T14:55:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:55:01.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-16T14:54:45.280907+0000', 'modified': '2024-01-16T14:54:48.302392+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.93:6836/2658521429', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6835', 'nonce': 2658521429}, {'type': 'v1', 'addr': '172.21.15.93:6836', 'nonce': 2658521429}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-16T14:55:01.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-16T14:55:01.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-16T14:55:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.293+0000 7f91ebd04700 1 Processor -- start 2024-01-16T14:55:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.294+0000 7f91ebd04700 1 -- start start 2024-01-16T14:55:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ebd04700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 0x7f91e406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ebd04700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91e406b510 0x7f91e406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ebd04700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e406b970 con 0x7f91e40d0080 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ebd04700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e4072cd0 con 0x7f91e406b510 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ebd04700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e4147bb0 con 0x7f91e4071bb0 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e9aa0700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91e40d0080 0x7f91e40b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58648/0 (socket says 172.21.15.26:58648) 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e9aa0700 1 -- 172.21.15.26:0/3601987720 learned_addr learned my addr 172.21.15.26:0/3601987720 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3601987720 >> v2:172.21.15.26:3300/0 conn(0x7f91e406b510 0x7f91e406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e929f700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 0x7f91e406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e8a9e700 1 -- 172.21.15.26:0/3601987720 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133824984 0 0) 0x7f91e406b970 con 0x7f91e40d0080 2024-01-16T14:55:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e8a9e700 1 -- 172.21.15.26:0/3601987720 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d00035b0 con 0x7f91e40d0080 2024-01-16T14:55:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3601987720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 msgr2=0x7f91e406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 0x7f91e406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3601987720 >> v1:172.21.15.26:6789/0 conn(0x7f91e40d0080 legacy=0x7f91e40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3601987720 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e40d0460 con 0x7f91e406b510 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91e929f700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 0x7f91e406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.295+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e406b510 0x7f91e406dfb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009d90 tx=0x7f91dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd144738a88a8f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91e8a9e700 1 -- 172.21.15.26:0/3601987720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00e070 con 0x7f91e406b510 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91e8a9e700 1 -- 172.21.15.26:0/3601987720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc00bb20 con 0x7f91e406b510 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91e8a9e700 1 -- 172.21.15.26:0/3601987720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00be90 con 0x7f91e406b510 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 -- 172.21.15.26:0/3601987720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e406b510 msgr2=0x7f91e406dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e406b510 0x7f91e406dfb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009d90 tx=0x7f91dc00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 -- 172.21.15.26:0/3601987720 shutdown_connections 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e406b510 0x7f91e406dfb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3601987720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e4071bb0 0x7f91e406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 -- 172.21.15.26:0/3601987720 >> 172.21.15.26:0/3601987720 conn(0x7f91e40b27d0 msgr2=0x7f91e40b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 -- 172.21.15.26:0/3601987720 shutdown_connections 2024-01-16T14:55:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.296+0000 7f91ebd04700 1 -- 172.21.15.26:0/3601987720 wait complete. 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.298+0000 7f91ebd04700 1 Processor -- start 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.298+0000 7f91ebd04700 1 -- start start 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.298+0000 7f91ebd04700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e406b510 0x7f91e415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.298+0000 7f91ebd04700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91e40d0080 0x7f91e412fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41e4af0 con 0x7f91e4071bb0 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41e3ee0 con 0x7f91e40d0080 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41e41c0 con 0x7f91e406b510 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55522/0 (socket says 172.21.15.26:55522) 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 -- 172.21.15.26:0/3630818982 learned_addr learned my addr 172.21.15.26:0/3630818982 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e406b510 0x7f91e415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91e40d0080 0x7f91e412fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 -- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e406b510 msgr2=0x7f91e415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e406b510 0x7f91e415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 -- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91e40d0080 msgr2=0x7f91e412fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91e40d0080 0x7f91e412fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 -- 172.21.15.26:0/3630818982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00094a0 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91e929f700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91e000ebe0 tx=0x7f91e000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91da7fc700 1 -- 172.21.15.26:0/3630818982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000c850 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91da7fc700 1 -- 172.21.15.26:0/3630818982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e000c9b0 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91da7fc700 1 -- 172.21.15.26:0/3630818982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000cd40 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc009a70 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.299+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e41e4780 con 0x7f91e4071bb0 2024-01-16T14:55:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.302+0000 7f91da7fc700 1 -- 172.21.15.26:0/3630818982 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f91e0013470 con 0x7f91e4071bb0 2024-01-16T14:55:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.302+0000 7f91da7fc700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 0x7f91c806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.302+0000 7f91da7fc700 1 -- 172.21.15.26:0/3630818982 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91e0089c20 con 0x7f91e4071bb0 2024-01-16T14:55:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.303+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 0x7f91c806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.303+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 0x7f91c806d810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91d4007fd0 tx=0x7f91d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.304+0000 7f91f7f00b80 1 Processor -- start 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.304+0000 7f91f7f00b80 1 -- start start 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bf9d0ce0 0x5584bfa8bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfa451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bf9b8480 con 0x5584bfa6d410 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bf99e770 con 0x5584bfa42dd0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584bf99c090 con 0x5584bf9d0ce0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91ea2a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfa451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55536/0 (socket says 172.21.15.26:55536) 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 -- 172.21.15.26:0/4213571570 learned_addr learned my addr 172.21.15.26:0/4213571570 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bf9d0ce0 0x5584bfa8bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 -- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bf9d0ce0 msgr2=0x5584bfa8bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bf9d0ce0 0x5584bfa8bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 -- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 msgr2=0x5584bfa451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfa451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 -- 172.21.15.26:0/4213571570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584bf7c0b10 con 0x5584bfa6d410 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfa451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91e929f700 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91e0060520 tx=0x7f91e00581b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba09b69fbf457645 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91c77fe700 1 -- 172.21.15.26:0/4213571570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0070070 con 0x5584bfa6d410 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91c77fe700 1 -- 172.21.15.26:0/4213571570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0058750 con 0x5584bfa6d410 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.305+0000 7f91c77fe700 1 -- 172.21.15.26:0/4213571570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e005c8a0 con 0x5584bfa6d410 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 -- 172.21.15.26:0/4213571570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 msgr2=0x5584bfa68190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f91e0060520 tx=0x7f91e00581b0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 -- 172.21.15.26:0/4213571570 shutdown_connections 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfa451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bf9d0ce0 0x5584bfa8bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/4213571570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa68190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 -- 172.21.15.26:0/4213571570 >> 172.21.15.26:0/4213571570 conn(0x5584bf972410 msgr2=0x5584bfa8a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 -- 172.21.15.26:0/4213571570 shutdown_connections 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.306+0000 7f91f7f00b80 1 -- 172.21.15.26:0/4213571570 wait complete. 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.307+0000 7f91f7f00b80 1 -- start start 2024-01-16T14:55:01.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.307+0000 7f91f7f00b80 1 Processor -- start 2024-01-16T14:55:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa71a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa6d410 0x5584bfa8bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bf997980 con 0x5584bf9d0ce0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bf937000 con 0x5584bfa6d410 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584bf936ca0 con 0x5584bfa42dd0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 10 client.0 _ll_register_callbacks cb 0x5584bf79a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55548/0 (socket says 172.21.15.26:55548) 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 -- 172.21.15.26:0/3643934930 learned_addr learned my addr 172.21.15.26:0/3643934930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa6d410 0x5584bfa8bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e929f700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa71a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 -- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 msgr2=0x5584bfa71a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa71a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 -- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa6d410 msgr2=0x5584bfa8bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa6d410 0x5584bfa8bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 -- 172.21.15.26:0/3643934930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e40d0460 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f91d40154b0 tx=0x7f91d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f380fcff64dccdd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4015b80 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91d4015ce0 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.309+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584bf9b7450 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d4015ea0 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 10 client.4593 fetch_fsmap learned FSMap version 8 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x7f91e41e4780 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5584bf9b7450 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91d4020070 con 0x5584bf9d0ce0 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 10 client.4593 fetch_fsmap finished waiting for FSMap version 8 2024-01-16T14:55:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 10 client.4593 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 20 client.4593 populate_metadata read hostname 'smithi026' 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa81fc0 0x5584bfa823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 4 client.4593 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 --> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584bf990960 con 0x5584bfa81fc0 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.310+0000 7f91e929f700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa81fc0 0x5584bfa823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.311+0000 7f91e929f700 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa81fc0 0x5584bfa823a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db26e8024217bd3 server_cookie=4a80351ca43e42c1 in_seq=0 out_seq=0 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.311+0000 7f91c4ff9700 10 client.4593 ms_handle_connect on v2:172.21.15.93:6835/2658521429 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.311+0000 7f91c4ff9700 1 -- 172.21.15.26:0/3643934930 <== mds.0 v2:172.21.15.93:6835/2658521429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584bf990960 con 0x5584bfa81fc0 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.311+0000 7f91c4ff9700 10 client.4593 handle_command_reply: tid=0 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.311+0000 7f91f7f00b80 1 client.4593 shutdown 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 msgr2=0x5584bfa65350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f91d40154b0 tx=0x7f91d4000bc0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 shutdown_connections 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa6d410 0x5584bfa8bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa65350 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa81fc0 0x5584bfa823a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3643934930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa71a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91c4ff9700 0 client.4593 ms_handle_reset on v2:172.21.15.93:6835/2658521429 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 >> 172.21.15.26:0/3643934930 conn(0x5584bf972410 msgr2=0x5584bfa44950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 shutdown_connections 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3643934930 wait complete. 2024-01-16T14:55:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.312+0000 7f91f7f00b80 20 client.4593 trim_cache size 0 max 16384 2024-01-16T14:55:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.331+0000 7f91f7f00b80 1 Processor -- start 2024-01-16T14:55:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.331+0000 7f91f7f00b80 1 -- start start 2024-01-16T14:55:01.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bf9d0ce0 0x5584bfa40130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bf816650 con 0x5584bfa6d410 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bfaac5a0 con 0x5584bf9d0ce0 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584bfa67db0 con 0x5584bfa42dd0 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55560/0 (socket says 172.21.15.26:55560) 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3759601118 learned_addr learned my addr 172.21.15.26:0/3759601118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bf9d0ce0 0x5584bfa40130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91e929f700 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 msgr2=0x5584bfa39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bf9d0ce0 msgr2=0x5584bfa40130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bf9d0ce0 0x5584bfa40130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 -- 172.21.15.26:0/3759601118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584bf959290 con 0x5584bfa6d410 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.332+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91e4068340 tx=0x7f91dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874f34d324ef9454 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91c7fff700 1 -- 172.21.15.26:0/3759601118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc021040 con 0x5584bfa6d410 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91c7fff700 1 -- 172.21.15.26:0/3759601118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc004240 con 0x5584bfa6d410 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91c7fff700 1 -- 172.21.15.26:0/3759601118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc0045f0 con 0x5584bfa6d410 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3759601118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 msgr2=0x5584bfa39d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f91e4068340 tx=0x7f91dc011000 comp rx=0 tx=0).stop 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3759601118 shutdown_connections 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bfa6d410 0x5584bfa39d70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bf9d0ce0 0x5584bfa40130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/3759601118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa42dd0 0x5584bfa39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3759601118 >> 172.21.15.26:0/3759601118 conn(0x5584bf972410 msgr2=0x5584bfa451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3759601118 shutdown_connections 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.333+0000 7f91f7f00b80 1 -- 172.21.15.26:0/3759601118 wait complete. 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.334+0000 7f91f7f00b80 1 -- start start 2024-01-16T14:55:01.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.334+0000 7f91f7f00b80 1 Processor -- start 2024-01-16T14:55:01.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfad94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa6d410 0x5584bfa39830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bfada4b0 con 0x5584bf9d0ce0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584bfa72890 con 0x5584bfa42dd0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584bfa726d0 con 0x5584bfa6d410 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91f7f00b80 10 client.0 _ll_register_callbacks cb 0x5584bf79a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55566/0 (socket says 172.21.15.26:55566) 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 -- 172.21.15.26:0/1722195730 learned_addr learned my addr 172.21.15.26:0/1722195730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e929f700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfad94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91ea2a1700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa6d410 0x5584bfa39830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 -- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa6d410 msgr2=0x5584bfa39830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa6d410 0x5584bfa39830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 -- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 msgr2=0x5584bfad94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfad94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.335+0000 7f91e9aa0700 1 -- 172.21.15.26:0/1722195730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc003b50 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91e9aa0700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000f60 tx=0x7f91d400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cea7998b83bfcf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40157c0 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91d4004a80 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91e0058750 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d4004c40 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 10 client.4599 fetch_fsmap learned FSMap version 8 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5584bf818c80 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e0058750 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91d403a020 con 0x5584bf9d0ce0 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 10 client.4599 fetch_fsmap finished waiting for FSMap version 8 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 10 client.4599 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-16T14:55:01.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 20 client.4599 populate_metadata read hostname 'smithi026' 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa80ad0 0x5584bfa709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 4 client.4599 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 --> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584bf990680 con 0x5584bfa80ad0 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.336+0000 7f91e929f700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa80ad0 0x5584bfa709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91e929f700 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa80ad0 0x5584bfa709d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b8aaa9e6b9cf44 server_cookie=2b761496fcabe887 in_seq=0 out_seq=0 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91c4ff9700 10 client.4599 ms_handle_connect on v2:172.21.15.93:6835/2658521429 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91c4ff9700 1 -- 172.21.15.26:0/1722195730 <== mds.0 v2:172.21.15.93:6835/2658521429 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5584bf990680 con 0x5584bfa80ad0 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91c4ff9700 10 client.4599 handle_command_reply: tid=0 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 client.4599 shutdown 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 msgr2=0x5584bfa72190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91d4000f60 tx=0x7f91d400a000 comp rx=0 tx=0).stop 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 shutdown_connections 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:6835/2658521429,v1:172.21.15.93:6836/2658521429] conn(0x5584bfa80ad0 0x5584bfa709d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5584bfa6d410 0x5584bfa39830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584bf9d0ce0 0x5584bfa72190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91c4ff9700 0 client.4599 ms_handle_reset on v2:172.21.15.93:6835/2658521429 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 --2- 172.21.15.26:0/1722195730 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5584bfa42dd0 0x5584bfad94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 >> 172.21.15.26:0/1722195730 conn(0x5584bf972410 msgr2=0x5584bfa451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 shutdown_connections 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 1 -- 172.21.15.26:0/1722195730 wait complete. 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.337+0000 7f91f7f00b80 20 client.4599 trim_cache size 0 max 16384 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "d39fe2d6-b323-4e7f-9f15-004380a58c0d", 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4282, 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-16T14:55:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 15.477022298, 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 8, 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 21, 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 21, 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 22.533742246999999 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:55:01.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 msgr2=0x7f91c806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 0x7f91c806d810 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91d4007fd0 tx=0x7f91d4009040 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 msgr2=0x7f91e41e4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91e000ebe0 tx=0x7f91e000eef0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 shutdown_connections 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91e40d0080 0x7f91e412fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91e406b510 0x7f91e415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f91c806b350 0x7f91c806d810 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 --2- 172.21.15.26:0/3630818982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91e4071bb0 0x7f91e41e4f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 >> 172.21.15.26:0/3630818982 conn(0x7f91e40b27d0 msgr2=0x7f91e40b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 shutdown_connections 2024-01-16T14:55:01.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.338+0000 7f91ebd04700 1 -- 172.21.15.26:0/3630818982 wait complete. 2024-01-16T14:55:01.334 INFO:teuthology.run_tasks:Running task kclient... 2024-01-16T14:55:01.356 INFO:tasks.kclient:Mounting kernel clients... 2024-01-16T14:55:01.356 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-16T14:55:01.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.356 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-16T14:55:01.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-01-16T14:55:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-01-16T14:55:01.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:55:01.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.453 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-16T14:55:01.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-01-16T14:55:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-01-16T14:55:01.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:55:01.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.594 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2024-01-16T14:55:01.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.615 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2024-01-16T14:55:01.688 INFO:teuthology.orchestra.run.smithi093.stderr:Cannot find device "ceph-brx" 2024-01-16T14:55:01.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:55:01.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.692 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2024-01-16T14:55:01.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:01.750 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2024-01-16T14:55:01.822 INFO:teuthology.orchestra.run.smithi093.stderr:Cannot find device "ceph-brx" 2024-01-16T14:55:01.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-16T14:55:01.824 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-01-16T14:55:01.824 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T14:55:01.824 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:01.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.987+0000 7fb13e68d700 1 Processor -- start 2024-01-16T14:55:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.987+0000 7fb13e68d700 1 -- start start 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13e68d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb1380d05e0 0x7fb1380b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13e68d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1380b0d40 0x7fb1380cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13e68d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13815c580 con 0x7fb13815c1a0 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13e68d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380cf450 con 0x7fb1380b0d40 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13e68d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380b46b0 con 0x7fb1380d05e0 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13cc2a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1380b0d40 0x7fb1380cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13cc2a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1380b0d40 0x7fb1380cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55580/0 (socket says 172.21.15.26:55580) 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.988+0000 7fb13cc2a700 1 -- 172.21.15.26:0/1185777538 learned_addr learned my addr 172.21.15.26:0/1185777538 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb1377fe700 1 --2- 172.21.15.26:0/1185777538 >> v2:172.21.15.93:3300/0 conn(0x7fb1380d05e0 0x7fb1380b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb13cc2a700 1 -- 172.21.15.26:0/1185777538 >> v2:172.21.15.93:3300/0 conn(0x7fb1380d05e0 msgr2=0x7fb1380b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb13cc2a700 1 --2- 172.21.15.26:0/1185777538 >> v2:172.21.15.93:3300/0 conn(0x7fb1380d05e0 0x7fb1380b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb13cc2a700 1 -- 172.21.15.26:0/1185777538 >> v1:172.21.15.93:6789/0 conn(0x7fb13815c1a0 legacy=0x7fb1380cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb13cc2a700 1 -- 172.21.15.26:0/1185777538 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380cf940 con 0x7fb1380b0d40 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb13cc2a700 1 --2- 172.21.15.26:0/1185777538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1380b0d40 0x7fb1380cc970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb13806ba00 tx=0x7fb13000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b61a603e22e66e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb136ffd700 1 -- 172.21.15.26:0/1185777538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000e070 con 0x7fb1380b0d40 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb136ffd700 1 -- 172.21.15.26:0/1185777538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13000b910 con 0x7fb1380b0d40 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.989+0000 7fb136ffd700 1 -- 172.21.15.26:0/1185777538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13000bcc0 con 0x7fb1380b0d40 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 -- 172.21.15.26:0/1185777538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1380b0d40 msgr2=0x7fb1380cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 --2- 172.21.15.26:0/1185777538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1380b0d40 0x7fb1380cc970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb13806ba00 tx=0x7fb13000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 -- 172.21.15.26:0/1185777538 shutdown_connections 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 --2- 172.21.15.26:0/1185777538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1380b0d40 0x7fb1380cc970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 --2- 172.21.15.26:0/1185777538 >> v2:172.21.15.93:3300/0 conn(0x7fb1380d05e0 0x7fb1380b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 -- 172.21.15.26:0/1185777538 >> 172.21.15.26:0/1185777538 conn(0x7fb1380b3800 msgr2=0x7fb1380b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 -- 172.21.15.26:0/1185777538 shutdown_connections 2024-01-16T14:55:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.990+0000 7fb13e68d700 1 -- 172.21.15.26:0/1185777538 wait complete. 2024-01-16T14:55:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.991+0000 7fb13e68d700 1 Processor -- start 2024-01-16T14:55:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.992+0000 7fb13e68d700 1 -- start start 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 0x7fb13806c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1380d05e0 0x7fb138154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1381e8460 con 0x7fb13815c1a0 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13806d880 con 0x7fb1380d05e0 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13e68d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13806c980 con 0x7fb1380b0d40 2024-01-16T14:55:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55590/0 (socket says 172.21.15.26:55590) 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 -- 172.21.15.26:0/2450091191 learned_addr learned my addr 172.21.15.26:0/2450091191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb1377fe700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1380d05e0 0x7fb138154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb137fff700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 0x7fb13806c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 -- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 msgr2=0x7fb13806c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 0x7fb13806c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 -- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1380d05e0 msgr2=0x7fb138154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1380d05e0 0x7fb138154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 -- 172.21.15.26:0/2450091191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb130009b30 con 0x7fb13815c1a0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb137fff700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 0x7fb13806c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.993+0000 7fb13cc2a700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb138071f70 tx=0x7fb130004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.994+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300048f0 con 0x7fb13815c1a0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.994+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1300099b0 con 0x7fb13815c1a0 2024-01-16T14:55:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.994+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13806cc00 con 0x7fb13815c1a0 2024-01-16T14:55:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.994+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13001b670 con 0x7fb13815c1a0 2024-01-16T14:55:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.994+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13806cf90 con 0x7fb13815c1a0 2024-01-16T14:55:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.995+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb138075480 con 0x7fb13815c1a0 2024-01-16T14:55:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.995+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb13001b7d0 con 0x7fb13815c1a0 2024-01-16T14:55:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.995+0000 7fb134ff9700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 0x7fb12406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.996+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb13008b0d0 con 0x7fb13815c1a0 2024-01-16T14:55:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.996+0000 7fb137fff700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 0x7fb12406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:01.996+0000 7fb137fff700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 0x7fb12406d810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb128000f60 tx=0x7fb128009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.001+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb130055fd0 con 0x7fb13815c1a0 2024-01-16T14:55:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.255+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb13806d560 con 0x7fb13815c1a0 2024-01-16T14:55:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.255+0000 7fb134ff9700 1 -- 172.21.15.26:0/2450091191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1300595f0 con 0x7fb13815c1a0 2024-01-16T14:55:02.238 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.257+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 msgr2=0x7fb12406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.257+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 0x7fb12406d810 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb128000f60 tx=0x7fb128009040 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.257+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 msgr2=0x7fb13815a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.257+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb138071f70 tx=0x7fb130004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 shutdown_connections 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb12406b350 0x7fb12406d810 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb13815c1a0 0x7fb13815a2d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1380b0d40 0x7fb13806c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 --2- 172.21.15.26:0/2450091191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1380d05e0 0x7fb138154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 >> 172.21.15.26:0/2450091191 conn(0x7fb1380b3800 msgr2=0x7fb1380cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 shutdown_connections 2024-01-16T14:55:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:02.258+0000 7fb13e68d700 1 -- 172.21.15.26:0/2450091191 wait complete. 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:02.255 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T14:55:02.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.255 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:02.275 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42423sec preferred_lft 42423sec 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:02.276 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-16T14:55:02.276 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-16T14:55:02.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-16T14:55:02.277 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:02.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.478 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:02.512 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-16T14:55:02.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.514 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-16T14:55:02.594 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-16T14:55:02.594 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:02.594 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-16T14:55:02.594 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-16T14:55:02.594 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:02.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-16T14:55:02.595 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:02.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.845 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-16T14:55:02.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.866 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-01-16T14:55:02.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:02.928 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:02.928 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:02.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T14:55:02.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-16T14:55:02.928 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:03.038 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-16T14:55:03.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-16T14:55:03.039 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:03.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.276 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:03.276 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:03.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-16T14:55:03.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-01-16T14:55:03.276 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:03.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:03.386 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:03.406 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-16T14:55:03.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:03.406 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:03.466 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:03.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:03.566 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:03.566 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:03.566 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:03.641 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:03.642 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:03.661 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:03.664 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:03.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:03.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.696 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:03.792 INFO:teuthology.orchestra.run.smithi026.stdout:client4648 2024-01-16T14:55:03.793 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:03.793 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:03.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.814 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:03.896 INFO:teuthology.orchestra.run.smithi026.stdout:client4648 2024-01-16T14:55:03.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.897 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:03.978 INFO:teuthology.orchestra.run.smithi026.stdout:client4648 2024-01-16T14:55:03.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:03.979 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d39fe2d6-b323-4e7f-9f15-004380a58c0d.client4648/status') 2024-01-16T14:55:04.071 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4648 (3)172.21.15.26:0/153772827 2024-01-16T14:55:04.071 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-01-16T14:55:04.072 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-01-16T14:55:04.072 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-01-16T14:55:04.072 INFO:teuthology.orchestra.run.smithi026.stderr:69 bytes copied, 3.0684e-05 s, 2.2 MB/s 2024-01-16T14:55:04.073 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-01-16T14:55:04.073 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T14:55:04.073 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-16T14:55:04.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.232+0000 7f774411a700 1 Processor -- start 2024-01-16T14:55:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.232+0000 7f774411a700 1 -- start start 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f774411a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c15c010 0x7f773c0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f774411a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f773c0ce560 0x7f773c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f774411a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c06f610 con 0x7f773c0cef20 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f774411a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0cf370 con 0x7f773c0ce560 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f774411a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0cd7f0 con 0x7f773c15c010 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77416b5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f773c0ce560 0x7f773c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77426b7700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f773c0cef20 0x7f773c0cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58660/0 (socket says 172.21.15.26:58660) 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77426b7700 1 -- 172.21.15.26:0/2563233107 learned_addr learned my addr 172.21.15.26:0/2563233107 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f7741eb6700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c15c010 0x7f773c0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77416b5700 1 -- 172.21.15.26:0/2563233107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c15c010 msgr2=0x7f773c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77416b5700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c15c010 0x7f773c0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77416b5700 1 -- 172.21.15.26:0/2563233107 >> v1:172.21.15.26:6789/0 conn(0x7f773c0cef20 legacy=0x7f773c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f7740eb4700 1 -- 172.21.15.26:0/2563233107 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881736652 0 0) 0x7f773c06f610 con 0x7f773c0cef20 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.233+0000 7f77416b5700 1 -- 172.21.15.26:0/2563233107 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0cdac0 con 0x7f773c0ce560 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f77416b5700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0ce560 0x7f773c0ce9e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f772c009840 tx=0x7f772c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ca345e8f7800e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f7740eb4700 1 -- 172.21.15.26:0/2563233107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011830 con 0x7f773c0ce560 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f7740eb4700 1 -- 172.21.15.26:0/2563233107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c011990 con 0x7f773c0ce560 2024-01-16T14:55:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f7740eb4700 1 -- 172.21.15.26:0/2563233107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c011d20 con 0x7f773c0ce560 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 -- 172.21.15.26:0/2563233107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0ce560 msgr2=0x7f773c0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0ce560 0x7f773c0ce9e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f772c009840 tx=0x7f772c009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 -- 172.21.15.26:0/2563233107 shutdown_connections 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c15c010 0x7f773c0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 --2- 172.21.15.26:0/2563233107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0ce560 0x7f773c0ce9e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.234+0000 7f774411a700 1 -- 172.21.15.26:0/2563233107 >> 172.21.15.26:0/2563233107 conn(0x7f773c0b3790 msgr2=0x7f773c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.235+0000 7f774411a700 1 -- 172.21.15.26:0/2563233107 shutdown_connections 2024-01-16T14:55:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.235+0000 7f774411a700 1 -- 172.21.15.26:0/2563233107 wait complete. 2024-01-16T14:55:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.236+0000 7f774411a700 1 Processor -- start 2024-01-16T14:55:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.236+0000 7f774411a700 1 -- start start 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c0ce560 0x7f773c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 0x7f773c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c7850 con 0x7f773c0cef20 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c6c40 con 0x7f773c15c010 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f774411a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c6f20 con 0x7f773c0ce560 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55608/0 (socket says 172.21.15.26:55608) 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 -- 172.21.15.26:0/3005106038 learned_addr learned my addr 172.21.15.26:0/3005106038 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f7741eb6700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c0ce560 0x7f773c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77426b7700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 0x7f773c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 -- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c0ce560 msgr2=0x7f773c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c0ce560 0x7f773c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 -- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 msgr2=0x7f773c146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 0x7f773c146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 -- 172.21.15.26:0/3005106038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0094a0 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77426b7700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 0x7f773c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.237+0000 7f77416b5700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f772c009fd0 tx=0x7f772c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.238+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0038a0 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.238+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c01f070 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.238+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0c7200 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.238+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c003c40 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.238+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c0b7650 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.239+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c0c7a00 con 0x7f773c0cef20 2024-01-16T14:55:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.239+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f772c026480 con 0x7f773c0cef20 2024-01-16T14:55:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.240+0000 7f77327fc700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 0x7f772006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.240+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f772c021070 con 0x7f773c0cef20 2024-01-16T14:55:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.240+0000 7f7741eb6700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 0x7f772006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:04.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.240+0000 7f7741eb6700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 0x7f772006d810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7738009ad0 tx=0x7f7738008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:04.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.245+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f773c0c7a00 con 0x7f773c0cef20 2024-01-16T14:55:04.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.511+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f773c069160 con 0x7f773c0cef20 2024-01-16T14:55:04.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.512+0000 7f77327fc700 1 -- 172.21.15.26:0/3005106038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f773c069160 con 0x7f773c0cef20 2024-01-16T14:55:04.495 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 msgr2=0x7f772006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 0x7f772006d810 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7738009ad0 tx=0x7f7738008040 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 msgr2=0x7f773c0b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f772c009fd0 tx=0x7f772c004750 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 shutdown_connections 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f773c15c010 0x7f773c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f773c0ce560 0x7f773c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f772006b350 0x7f772006d810 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 --2- 172.21.15.26:0/3005106038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f773c0cef20 0x7f773c0b7110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 >> 172.21.15.26:0/3005106038 conn(0x7f773c0b3790 msgr2=0x7f773c0d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 shutdown_connections 2024-01-16T14:55:04.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:04.514+0000 7f774411a700 1 -- 172.21.15.26:0/3005106038 wait complete. 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T14:55:04.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:04.511 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-16T14:55:04.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.511 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:04.530 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.531 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:9e:c4:45:96:b6 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::b438:a1ff:fe82:ed32/64 scope link 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:9e:c4:45:96:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c49e:c4ff:fe45:96b6/64 scope link tentative 2024-01-16T14:55:04.532 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:04.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.533 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:04.602 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-16T14:55:04.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.604 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:04.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.669 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:04.669 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:04.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-16T14:55:04.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-16T14:55:04.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-16T14:55:04.670 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:04.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.794 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-16T14:55:04.815 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-16T14:55:04.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.815 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-01-16T14:55:04.877 INFO:teuthology.orchestra.run.smithi026.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-16T14:55:04.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.878 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:04.878 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:04.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-16T14:55:04.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-16T14:55:04.878 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:04.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:04.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-16T14:55:05.031 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:05.031 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether e2:b3:81:2a:b4:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T14:55:05.032 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::e0b3:81ff:fe2a:b4c2/64 scope link 2024-01-16T14:55:05.033 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether e2:b3:81:2a:b4:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T14:55:05.127 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T14:55:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::e0b3:81ff:fe2a:b4c2/64 scope link 2024-01-16T14:55:05.128 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:05.140 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-16T14:55:05.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-16T14:55:05.141 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:05.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.393 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:05.393 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:05.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.1 up 2024-01-16T14:55:05.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.1 master ceph-brx 2024-01-16T14:55:05.393 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:05.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:05.507 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-16T14:55:05.525 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-16T14:55:05.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:05.526 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-16T14:55:05.586 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:05.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T14:55:05.686 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T14:55:05.686 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T14:55:05.686 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:05.696 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T14:55:05.696 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T14:55:05.696 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:05.696 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:05.697 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:05.716 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:05.717 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:05.720 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:05.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-16T14:55:05.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.752 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:05.835 INFO:teuthology.orchestra.run.smithi026.stdout:client4484 2024-01-16T14:55:05.835 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:05.835 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:05.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.899 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:05.983 INFO:teuthology.orchestra.run.smithi026.stdout:client4484 2024-01-16T14:55:05.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:05.984 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:06.066 INFO:teuthology.orchestra.run.smithi026.stdout:client4484 2024-01-16T14:55:06.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.067 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d39fe2d6-b323-4e7f-9f15-004380a58c0d.client4484/status') 2024-01-16T14:55:06.160 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4484 (3)172.21.15.26:0/2898659731 2024-01-16T14:55:06.160 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2024-01-16T14:55:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2024-01-16T14:55:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2024-01-16T14:55:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 2.6124e-05 s, 2.7 MB/s 2024-01-16T14:55:06.162 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-01-16T14:55:06.162 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T14:55:06.162 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:06.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:06.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.322+0000 7f2b32873700 1 Processor -- start 2024-01-16T14:55:06.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.322+0000 7f2b32873700 1 -- start start 2024-01-16T14:55:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b32873700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2c1457a0 0x7f2b2c0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b32873700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2b2c06afd0 0x7f2b2c06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b32873700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0d04b0 con 0x7f2b2c0cd470 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b32873700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c06b8f0 con 0x7f2b2c1457a0 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b32873700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0cf1b0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2c1457a0 0x7f2b2c0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2c1457a0 0x7f2b2c0cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41194/0 (socket says 172.21.15.26:41194) 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 -- 172.21.15.26:0/3227825753 learned_addr learned my addr 172.21.15.26:0/3227825753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b30e10700 1 --2- 172.21.15.26:0/3227825753 >> v2:172.21.15.93:3300/0 conn(0x7f2b2c06afd0 0x7f2b2c06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 -- 172.21.15.26:0/3227825753 >> v2:172.21.15.93:3300/0 conn(0x7f2b2c06afd0 msgr2=0x7f2b2c06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:06.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 --2- 172.21.15.26:0/3227825753 >> v2:172.21.15.93:3300/0 conn(0x7f2b2c06afd0 0x7f2b2c06b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 -- 172.21.15.26:0/3227825753 >> v1:172.21.15.93:6789/0 conn(0x7f2b2c0cd470 legacy=0x7f2b2c160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 -- 172.21.15.26:0/3227825753 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0cf490 con 0x7f2b2c1457a0 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.323+0000 7f2b2bfff700 1 --2- 172.21.15.26:0/3227825753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c1457a0 0x7f2b2c0cff70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c072f50 tx=0x7f2b1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527ca5e7a0eec6a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b2affd700 1 -- 172.21.15.26:0/3227825753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011830 con 0x7f2b2c1457a0 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b2affd700 1 -- 172.21.15.26:0/3227825753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c011990 con 0x7f2b2c1457a0 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b2affd700 1 -- 172.21.15.26:0/3227825753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011d20 con 0x7f2b2c1457a0 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 -- 172.21.15.26:0/3227825753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c1457a0 msgr2=0x7f2b2c0cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 --2- 172.21.15.26:0/3227825753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c1457a0 0x7f2b2c0cff70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c072f50 tx=0x7f2b1c009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 -- 172.21.15.26:0/3227825753 shutdown_connections 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 --2- 172.21.15.26:0/3227825753 >> v2:172.21.15.93:3300/0 conn(0x7f2b2c06afd0 0x7f2b2c06b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 --2- 172.21.15.26:0/3227825753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c1457a0 0x7f2b2c0cff70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 -- 172.21.15.26:0/3227825753 >> 172.21.15.26:0/3227825753 conn(0x7f2b2c0d0aa0 msgr2=0x7f2b2c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:06.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 -- 172.21.15.26:0/3227825753 shutdown_connections 2024-01-16T14:55:06.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.324+0000 7f2b32873700 1 -- 172.21.15.26:0/3227825753 wait complete. 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.326+0000 7f2b32873700 1 Processor -- start 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.326+0000 7f2b32873700 1 -- start start 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.326+0000 7f2b32873700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b2c0cd470 0x7f2b2c0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b2c1457a0 0x7f2b2c1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0a9bc0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c15bb60 con 0x7f2b2c0cd470 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0a9990 con 0x7f2b2c1457a0 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41206/0 (socket says 172.21.15.26:41206) 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 -- 172.21.15.26:0/2753409754 learned_addr learned my addr 172.21.15.26:0/2753409754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2b7fe700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b2c0cd470 0x7f2b2c0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b30e10700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b2c1457a0 0x7f2b2c1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 -- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b2c1457a0 msgr2=0x7f2b2c1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b2c1457a0 0x7f2b2c1f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 -- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b2c0cd470 msgr2=0x7f2b2c0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b2c0cd470 0x7f2b2c0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 -- 172.21.15.26:0/2753409754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009550 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b2bfff700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009c10 tx=0x7f2b1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c004380 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c01f070 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c0036a0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.327+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c13ede0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.328+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c13f3a0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.328+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c075480 con 0x7f2b2c06afd0 2024-01-16T14:55:06.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.329+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2b1c021030 con 0x7f2b2c06afd0 2024-01-16T14:55:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.329+0000 7f2b28ff9700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 0x7f2b1806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.329+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b1c08aec0 con 0x7f2b2c06afd0 2024-01-16T14:55:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.330+0000 7f2b2b7fe700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 0x7f2b1806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.330+0000 7f2b2b7fe700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 0x7f2b1806d810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c15b810 tx=0x7f2b20009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.334+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b1c055d40 con 0x7f2b2c06afd0 2024-01-16T14:55:06.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.596+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b2c0d0880 con 0x7f2b2c06afd0 2024-01-16T14:55:06.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.596+0000 7f2b28ff9700 1 -- 172.21.15.26:0/2753409754 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b1c059360 con 0x7f2b2c06afd0 2024-01-16T14:55:06.579 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 msgr2=0x7f2b1806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 0x7f2b1806d810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c15b810 tx=0x7f2b20009040 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 msgr2=0x7f2b2c0a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009c10 tx=0x7f2b1c004750 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 shutdown_connections 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b2c1457a0 0x7f2b2c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2b1806b350 0x7f2b1806d810 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2c06afd0 0x7f2b2c0a9450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 --2- 172.21.15.26:0/2753409754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b2c0cd470 0x7f2b2c0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.598+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 >> 172.21.15.26:0/2753409754 conn(0x7f2b2c0d0aa0 msgr2=0x7f2b2c06c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.599+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 shutdown_connections 2024-01-16T14:55:06.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:06.599+0000 7f2b32873700 1 -- 172.21.15.26:0/2753409754 wait complete. 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T14:55:06.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T14:55:06.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:06.596 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T14:55:06.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.596 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2024-01-16T14:55:06.619 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:06.619 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:06.620 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:06.621 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T14:55:06.621 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42451sec preferred_lft 42451sec 2024-01-16T14:55:06.621 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::fd4b:da16:302c:bf58/64 scope link noprefixroute 2024-01-16T14:55:06.621 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:06.621 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-16T14:55:06.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-16T14:55:06.621 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:06.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.794 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:06.822 INFO:teuthology.orchestra.run.smithi093.stdout:1 2024-01-16T14:55:06.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.824 DEBUG:teuthology.orchestra.run.smithi093:> route 2024-01-16T14:55:06.919 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2024-01-16T14:55:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T14:55:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T14:55:06.920 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:06.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:06.920 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:06.920 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:06.920 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T14:55:06.920 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T14:55:06.921 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T14:55:06.921 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:07.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.090 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2024-01-16T14:55:07.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.107 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2024-01-16T14:55:07.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.164 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:07.164 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:07.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T14:55:07.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-16T14:55:07.164 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:07.258 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-16T14:55:07.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.258 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:07.258 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:07.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-16T14:55:07.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-16T14:55:07.259 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-16T14:55:07.259 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-16T14:55:07.259 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-16T14:55:07.259 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:07.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.466 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:07.467 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:07.467 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2024-01-16T14:55:07.467 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2024-01-16T14:55:07.467 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:07.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:07.565 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:07.581 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-16T14:55:07.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:07.582 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:07.638 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:07.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.638 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:07.725 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:07.725 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:07.725 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:07.816 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:07.817 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:07.817 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:07.817 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:07.817 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:07.835 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:07.836 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:07.838 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:07.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.839 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:07.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.865 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:07.937 INFO:teuthology.orchestra.run.smithi093.stdout:client4623 2024-01-16T14:55:07.937 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:07.937 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:07.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:07.995 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:08.066 INFO:teuthology.orchestra.run.smithi093.stdout:client4623 2024-01-16T14:55:08.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.067 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:client4623 2024-01-16T14:55:08.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.138 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d39fe2d6-b323-4e7f-9f15-004380a58c0d.client4623/status') 2024-01-16T14:55:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:instance: client.4623 (3)172.21.15.93:0/1541013371 2024-01-16T14:55:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:blocklisted: false 2024-01-16T14:55:08.220 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records in 2024-01-16T14:55:08.220 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records out 2024-01-16T14:55:08.220 INFO:teuthology.orchestra.run.smithi093.stderr:70 bytes copied, 3.6762e-05 s, 1.9 MB/s 2024-01-16T14:55:08.221 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-01-16T14:55:08.221 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-16T14:55:08.221 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-16T14:55:08.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:08.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.385+0000 7ff9ea393700 1 Processor -- start 2024-01-16T14:55:08.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.385+0000 7ff9ea393700 1 -- start start 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9ea393700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9e415e000 0x7ff9e406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9ea393700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e40cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9ea393700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e40cfd60 con 0x7ff9e40d0ae0 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9ea393700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e40cf4b0 con 0x7ff9e415e000 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9ea393700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e406f430 con 0x7ff9e406fe00 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9e3fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9e415e000 0x7ff9e406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9e3fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9e415e000 0x7ff9e406eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41214/0 (socket says 172.21.15.26:41214) 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9e3fff700 1 -- 172.21.15.26:0/3072632371 learned_addr learned my addr 172.21.15.26:0/3072632371 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.386+0000 7ff9e8930700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e40cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e3fff700 1 -- 172.21.15.26:0/3072632371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 msgr2=0x7ff9e40cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e3fff700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e40cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e3fff700 1 -- 172.21.15.26:0/3072632371 >> v1:172.21.15.93:6789/0 conn(0x7ff9e40d0ae0 legacy=0x7ff9e406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e3fff700 1 -- 172.21.15.26:0/3072632371 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40cf1d0 con 0x7ff9e415e000 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e8930700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e40cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e3fff700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e406eef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40098f0 tx=0x7ff9d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159cbbf9f37f9c16 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e2ffd700 1 -- 172.21.15.26:0/3072632371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400b8a0 con 0x7ff9e415e000 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e2ffd700 1 -- 172.21.15.26:0/3072632371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d400ba00 con 0x7ff9e415e000 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.387+0000 7ff9e2ffd700 1 -- 172.21.15.26:0/3072632371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400bd90 con 0x7ff9e415e000 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 -- 172.21.15.26:0/3072632371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 msgr2=0x7ff9e406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e406eef0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40098f0 tx=0x7ff9d4009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 -- 172.21.15.26:0/3072632371 shutdown_connections 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e40cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 --2- 172.21.15.26:0/3072632371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e406eef0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 -- 172.21.15.26:0/3072632371 >> 172.21.15.26:0/3072632371 conn(0x7ff9e40b3720 msgr2=0x7ff9e40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 -- 172.21.15.26:0/3072632371 shutdown_connections 2024-01-16T14:55:08.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.388+0000 7ff9ea393700 1 -- 172.21.15.26:0/3072632371 wait complete. 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.389+0000 7ff9ea393700 1 Processor -- start 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.389+0000 7ff9ea393700 1 -- start start 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e4153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9e40d0ae0 0x7ff9e415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e4153940 con 0x7ff9e415e000 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e41f5340 con 0x7ff9e40d0ae0 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9ea393700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e40b6c90 con 0x7ff9e406fe00 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e8930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e3fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e4153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e8930700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41222/0 (socket says 172.21.15.26:41222) 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e8930700 1 -- 172.21.15.26:0/2930639430 learned_addr learned my addr 172.21.15.26:0/2930639430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:08.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e37fe700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9e40d0ae0 0x7ff9e415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e3fff700 1 -- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9e40d0ae0 msgr2=0x7ff9e415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e3fff700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9e40d0ae0 0x7ff9e415b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e3fff700 1 -- 172.21.15.26:0/2930639430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 msgr2=0x7ff9e415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.390+0000 7ff9e3fff700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e3fff700 1 -- 172.21.15.26:0/2930639430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d4009550 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e8930700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e3fff700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e4153400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4016000 tx=0x7ff9d4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004300 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d401f070 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e40b69b0 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40036a0 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.391+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e41f4fd0 con 0x7ff9e406fe00 2024-01-16T14:55:08.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.392+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e41f5160 con 0x7ff9e406fe00 2024-01-16T14:55:08.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.392+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff9d4021030 con 0x7ff9e406fe00 2024-01-16T14:55:08.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.393+0000 7ff9e0ff9700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 0x7ff9d006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:08.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.393+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9d408ae40 con 0x7ff9e406fe00 2024-01-16T14:55:08.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.393+0000 7ff9e37fe700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 0x7ff9d006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:08.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.394+0000 7ff9e37fe700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 0x7ff9d006d810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9e406d620 tx=0x7ff9d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:08.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.398+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9d4055ca0 con 0x7ff9e406fe00 2024-01-16T14:55:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.658+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9e4075480 con 0x7ff9e406fe00 2024-01-16T14:55:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.659+0000 7ff9e0ff9700 1 -- 172.21.15.26:0/2930639430 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9d40592c0 con 0x7ff9e406fe00 2024-01-16T14:55:08.642 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 msgr2=0x7ff9d006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 0x7ff9d006d810 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9e406d620 tx=0x7ff9d8009040 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 msgr2=0x7ff9e4153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e4153400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4016000 tx=0x7ff9d4004750 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 shutdown_connections 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff9d006b350 0x7ff9d006d810 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9e415e000 0x7ff9e415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9e406fe00 0x7ff9e4153400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 --2- 172.21.15.26:0/2930639430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9e40d0ae0 0x7ff9e415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 >> 172.21.15.26:0/2930639430 conn(0x7ff9e40b3720 msgr2=0x7ff9e40b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 shutdown_connections 2024-01-16T14:55:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:08.661+0000 7ff9ea393700 1 -- 172.21.15.26:0/2930639430 wait complete. 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-16T14:55:08.658 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:08.659 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-16T14:55:08.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.659 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:08.677 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42449sec preferred_lft 42449sec 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::fd4b:da16:302c:bf58/64 scope link noprefixroute 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether da:53:f5:87:db:ed brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::2432:a9ff:fe27:8dbe/64 scope link 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.678 INFO:teuthology.orchestra.run.smithi093.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:08.679 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether da:53:f5:87:db:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T14:55:08.679 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::d853:f5ff:fe87:dbed/64 scope link tentative 2024-01-16T14:55:08.679 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:08.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.679 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:08.742 INFO:teuthology.orchestra.run.smithi093.stdout:1 2024-01-16T14:55:08.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.743 DEBUG:teuthology.orchestra.run.smithi093:> route 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:08.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T14:55:08.804 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:08.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.908 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2024-01-16T14:55:08.925 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-16T14:55:08.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.925 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2024-01-16T14:55:08.981 INFO:teuthology.orchestra.run.smithi093.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-16T14:55:08.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:08.981 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:08.982 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:08.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-16T14:55:08.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-16T14:55:08.982 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:09.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.076 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-16T14:55:09.117 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 82:9d:f6:b1:f3:6c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::809d:f6ff:feb1:f36c/64 scope link 2024-01-16T14:55:09.118 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.137 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-16T14:55:09.214 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 82:9d:f6:b1:f3:6c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::809d:f6ff:feb1:f36c/64 scope link 2024-01-16T14:55:09.215 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:09.231 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-16T14:55:09.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-16T14:55:09.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-16T14:55:09.232 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:09.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.453 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:09.453 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:09.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.1 up 2024-01-16T14:55:09.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.1 master ceph-brx 2024-01-16T14:55:09.453 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:09.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:09.552 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-16T14:55:09.569 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-16T14:55:09.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:09.569 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-16T14:55:09.626 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:09.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.627 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T14:55:09.713 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T14:55:09.713 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T14:55:09.713 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:09.722 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T14:55:09.722 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T14:55:09.722 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:09.722 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:09.722 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-01-16T14:55:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-01-16T14:55:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:09.742 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:09.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.743 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-16T14:55:09.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.769 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:09.841 INFO:teuthology.orchestra.run.smithi093.stdout:client4632 2024-01-16T14:55:09.841 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:09.841 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:09.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.898 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:09.971 INFO:teuthology.orchestra.run.smithi093.stdout:client4632 2024-01-16T14:55:09.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:09.971 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:10.043 INFO:teuthology.orchestra.run.smithi093.stdout:client4632 2024-01-16T14:55:10.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:10.043 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d39fe2d6-b323-4e7f-9f15-004380a58c0d.client4632/status') 2024-01-16T14:55:10.123 INFO:teuthology.orchestra.run.smithi093.stdout:instance: client.4632 (3)172.21.15.93:0/3073057572 2024-01-16T14:55:10.123 INFO:teuthology.orchestra.run.smithi093.stdout:blocklisted: false 2024-01-16T14:55:10.123 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records in 2024-01-16T14:55:10.124 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records out 2024-01-16T14:55:10.124 INFO:teuthology.orchestra.run.smithi093.stderr:70 bytes copied, 2.1458e-05 s, 3.3 MB/s 2024-01-16T14:55:10.125 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-16T14:55:10.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:10.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.325+0000 7f6690580700 1 Processor -- start 2024-01-16T14:55:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.325+0000 7f6690580700 1 -- start start 2024-01-16T14:55:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f6690580700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f668815c1c0 0x7f66880cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f6690580700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f6690580700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66880cc670 con 0x7f668815bde0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f6690580700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880cffb0 con 0x7f66880b4240 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f6690580700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66880cf7a0 con 0x7f668815c1c0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668eb1d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668db1b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f668815c1c0 0x7f66880cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668eb1d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41224/0 (socket says 172.21.15.26:41224) 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668eb1d700 1 -- 172.21.15.26:0/2339866295 learned_addr learned my addr 172.21.15.26:0/2339866295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668db1b700 1 -- 172.21.15.26:0/2339866295 >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 msgr2=0x7f66880cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668db1b700 1 --2- 172.21.15.26:0/2339866295 >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668db1b700 1 -- 172.21.15.26:0/2339866295 >> v1:172.21.15.93:6789/0 conn(0x7f668815bde0 legacy=0x7f668806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668db1b700 1 -- 172.21.15.26:0/2339866295 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880cdb20 con 0x7f668815c1c0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668d31a700 1 -- 172.21.15.26:0/2339866295 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684866677 0 0) 0x7f66880cc670 con 0x7f668815bde0 2024-01-16T14:55:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.326+0000 7f668eb1d700 1 --2- 172.21.15.26:0/2339866295 >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f668db1b700 1 --2- 172.21.15.26:0/2339866295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c1c0 0x7f66880cd0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6688071ce0 tx=0x7f6678009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e617db80b83cf97 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f668d31a700 1 -- 172.21.15.26:0/2339866295 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678011830 con 0x7f668815c1c0 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f668d31a700 1 -- 172.21.15.26:0/2339866295 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6678011990 con 0x7f668815c1c0 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f668d31a700 1 -- 172.21.15.26:0/2339866295 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678011d20 con 0x7f668815c1c0 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 -- 172.21.15.26:0/2339866295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c1c0 msgr2=0x7f66880cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 --2- 172.21.15.26:0/2339866295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c1c0 0x7f66880cd0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6688071ce0 tx=0x7f6678009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 -- 172.21.15.26:0/2339866295 shutdown_connections 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 --2- 172.21.15.26:0/2339866295 >> v2:172.21.15.26:3300/0 conn(0x7f66880b4240 0x7f66880cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 --2- 172.21.15.26:0/2339866295 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c1c0 0x7f66880cd0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 -- 172.21.15.26:0/2339866295 >> 172.21.15.26:0/2339866295 conn(0x7f66880b3720 msgr2=0x7f66880b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 -- 172.21.15.26:0/2339866295 shutdown_connections 2024-01-16T14:55:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.327+0000 7f6690580700 1 -- 172.21.15.26:0/2339866295 wait complete. 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.329+0000 7f6690580700 1 Processor -- start 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.329+0000 7f6690580700 1 -- start start 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.329+0000 7f6690580700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.329+0000 7f6690580700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f668815bde0 0x7f66880c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f6690580700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c5b0 0x7f6688159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f6690580700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66881536c0 con 0x7f66880b4240 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f6690580700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668815ca40 con 0x7f668815c5b0 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f6690580700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668815cd20 con 0x7f668815bde0 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41226/0 (socket says 172.21.15.26:41226) 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 -- 172.21.15.26:0/939602290 learned_addr learned my addr 172.21.15.26:0/939602290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668eb1d700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c5b0 0x7f6688159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668db1b700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f668815bde0 0x7f66880c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 -- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f668815bde0 msgr2=0x7f66880c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f668815bde0 0x7f66880c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 -- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c5b0 msgr2=0x7f6688159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c5b0 0x7f6688159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 -- 172.21.15.26:0/939602290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009b30 con 0x7f66880b4240 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f668e31c700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66840011e0 tx=0x7f668400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840054e0 con 0x7f66880b4240 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684003680 con 0x7f66880b4240 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66780094a0 con 0x7f66880b4240 2024-01-16T14:55:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.330+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684003a30 con 0x7f66880b4240 2024-01-16T14:55:10.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.331+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668815d2e0 con 0x7f66880b4240 2024-01-16T14:55:10.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.332+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6688075480 con 0x7f66880b4240 2024-01-16T14:55:10.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.332+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6684003b90 con 0x7f66880b4240 2024-01-16T14:55:10.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.333+0000 7f667effd700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 0x7f666c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.333+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6684089050 con 0x7f66880b4240 2024-01-16T14:55:10.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.333+0000 7f668db1b700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 0x7f666c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.333+0000 7f668db1b700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 0x7f666c06d810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66780090b0 tx=0x7f667801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.338+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6684053f50 con 0x7f66880b4240 2024-01-16T14:55:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.550+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f668815adf0 con 0x7f66880b4240 2024-01-16T14:55:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.551+0000 7f667effd700 1 -- 172.21.15.26:0/939602290 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f6684017020 con 0x7f66880b4240 2024-01-16T14:55:10.535 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:10.535 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:10.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 msgr2=0x7f666c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 0x7f666c06d810 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66780090b0 tx=0x7f667801c040 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 msgr2=0x7f6688153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66840011e0 tx=0x7f668400cae0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 shutdown_connections 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668815c5b0 0x7f6688159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f666c06b350 0x7f666c06d810 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66880b4240 0x7f6688153180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 --2- 172.21.15.26:0/939602290 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f668815bde0 0x7f66880c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 >> 172.21.15.26:0/939602290 conn(0x7f66880b3720 msgr2=0x7f66880b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 shutdown_connections 2024-01-16T14:55:10.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.553+0000 7f6690580700 1 -- 172.21.15.26:0/939602290 wait complete. 2024-01-16T14:55:10.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:10.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:10.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.707+0000 7f1bf6c80700 1 Processor -- start 2024-01-16T14:55:10.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.707+0000 7f1bf6c80700 1 -- start start 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf6c80700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bf015bde0 0x7f1bf015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf6c80700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 0x7f1bf00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf6c80700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf00d0a90 con 0x7f1bf00cf2b0 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf6c80700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf00cefb0 con 0x7f1bf015bde0 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf6c80700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf006ee10 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1bf4a1c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1bf015bde0 0x7f1bf015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1beffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1bf00cf2b0 0x7f1bf00d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59520/0 (socket says 172.21.15.26:59520) 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.708+0000 7f1beffff700 1 -- 172.21.15.26:0/2150695100 learned_addr learned my addr 172.21.15.26:0/2150695100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf521d700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 0x7f1bf00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/2150695100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 msgr2=0x7f1bf00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf4a1c700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 0x7f1bf00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/2150695100 >> v1:172.21.15.26:6789/0 conn(0x7f1bf00cf2b0 legacy=0x7f1bf00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/2150695100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf006f0f0 con 0x7f1bf015bde0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bef7fe700 1 -- 172.21.15.26:0/2150695100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625021980 0 0) 0x7f1bf00d0a90 con 0x7f1bf00cf2b0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf521d700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 0x7f1bf00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bf4a1c700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf015bde0 0x7f1bf015c1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009840 tx=0x7f1be0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3322511257689e99 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bef7fe700 1 -- 172.21.15.26:0/2150695100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0011830 con 0x7f1bf015bde0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bef7fe700 1 -- 172.21.15.26:0/2150695100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be0011990 con 0x7f1bf015bde0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.709+0000 7f1bef7fe700 1 -- 172.21.15.26:0/2150695100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0011d20 con 0x7f1bf015bde0 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 -- 172.21.15.26:0/2150695100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf015bde0 msgr2=0x7f1bf015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf015bde0 0x7f1bf015c1c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009840 tx=0x7f1be0009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 -- 172.21.15.26:0/2150695100 shutdown_connections 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf00ce5c0 0x7f1bf00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/2150695100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf015bde0 0x7f1bf015c1c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 -- 172.21.15.26:0/2150695100 >> 172.21.15.26:0/2150695100 conn(0x7f1bf00b3720 msgr2=0x7f1bf00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:10.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 -- 172.21.15.26:0/2150695100 shutdown_connections 2024-01-16T14:55:10.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.710+0000 7f1bf6c80700 1 -- 172.21.15.26:0/2150695100 wait complete. 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.711+0000 7f1bf6c80700 1 Processor -- start 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.711+0000 7f1bf6c80700 1 -- start start 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bf00cf2b0 0x7f1bf015dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf015bde0 0x7f1bf015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf006d530 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf006d9c0 con 0x7f1bf00cf2b0 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf6c80700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf006cc00 con 0x7f1bf015bde0 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf4a1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf4a1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41244/0 (socket says 172.21.15.26:41244) 2024-01-16T14:55:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/955946502 learned_addr learned my addr 172.21.15.26:0/955946502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1bf521d700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf015bde0 0x7f1bf015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.712+0000 7f1beffff700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bf00cf2b0 0x7f1bf015dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf015bde0 msgr2=0x7f1bf015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf015bde0 0x7f1bf015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bf00cf2b0 msgr2=0x7f1bf015dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bf00cf2b0 0x7f1bf015dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 -- 172.21.15.26:0/955946502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00094a0 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf4a1c700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000c00 tx=0x7f1be0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0004550 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be001e070 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf006cee0 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be00038e0 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.713+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf01f6a70 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.714+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf0075480 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.715+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1be0003a40 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.715+0000 7f1bed7fa700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 0x7f1bdc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.715+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be008aa80 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.715+0000 7f1beffff700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 0x7f1bdc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.716+0000 7f1beffff700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 0x7f1bdc06d810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00b7810 tx=0x7f1be4007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:10.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.720+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1be0055980 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.931+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bf015f4f0 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.931+0000 7f1bed7fa700 1 -- 172.21.15.26:0/955946502 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1be00150a0 con 0x7f1bf00ce5c0 2024-01-16T14:55:10.915 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 msgr2=0x7f1bdc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 0x7f1bdc06d810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00b7810 tx=0x7f1be4007de0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 msgr2=0x7f1bf01f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000c00 tx=0x7f1be0004750 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 shutdown_connections 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1bf015bde0 0x7f1bf015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1bdc06b350 0x7f1bdc06d810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bf00ce5c0 0x7f1bf01f6530 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 --2- 172.21.15.26:0/955946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1bf00cf2b0 0x7f1bf015dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 >> 172.21.15.26:0/955946502 conn(0x7f1bf00b3720 msgr2=0x7f1bf00b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 shutdown_connections 2024-01-16T14:55:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:10.933+0000 7f1bf6c80700 1 -- 172.21.15.26:0/955946502 wait complete. 2024-01-16T14:55:10.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:11.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.086+0000 7f4d6bc27700 1 Processor -- start 2024-01-16T14:55:11.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.086+0000 7f4d6bc27700 1 -- start start 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6bc27700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 0x7f4d64066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6bc27700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d6406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6bc27700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6406b680 con 0x7f4d6415a600 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6bc27700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640cc680 con 0x7f4d6406c6b0 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6bc27700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6406b070 con 0x7f4d640b3e70 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d691c2700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4d6415a600 0x7f4d640ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59536/0 (socket says 172.21.15.26:59536) 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d691c2700 1 -- 172.21.15.26:0/554865782 learned_addr learned my addr 172.21.15.26:0/554865782 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d699c3700 1 --2- 172.21.15.26:0/554865782 >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 0x7f4d64066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.087+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/554865782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d6406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d689c1700 1 -- 172.21.15.26:0/554865782 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970178352 0 0) 0x7f4d6406b680 con 0x7f4d6415a600 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d689c1700 1 -- 172.21.15.26:0/554865782 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d500035b0 con 0x7f4d6415a600 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/554865782 >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 msgr2=0x7f4d64066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/554865782 >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 0x7f4d64066ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/554865782 >> v1:172.21.15.26:6789/0 conn(0x7f4d6415a600 legacy=0x7f4d640ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/554865782 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6406b310 con 0x7f4d640b3e70 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d699c3700 1 --2- 172.21.15.26:0/554865782 >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 0x7f4d64066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/554865782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d6406bb10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009d90 tx=0x7f4d5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52c66acb7634e1a server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d689c1700 1 -- 172.21.15.26:0/554865782 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c00e070 con 0x7f4d640b3e70 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d689c1700 1 -- 172.21.15.26:0/554865782 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00bb20 con 0x7f4d640b3e70 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.088+0000 7f4d689c1700 1 -- 172.21.15.26:0/554865782 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c012430 con 0x7f4d640b3e70 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 -- 172.21.15.26:0/554865782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 msgr2=0x7f4d6406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/554865782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d6406bb10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009d90 tx=0x7f4d5c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 -- 172.21.15.26:0/554865782 shutdown_connections 2024-01-16T14:55:11.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/554865782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d6406bb10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/554865782 >> v2:172.21.15.93:3300/0 conn(0x7f4d6406c6b0 0x7f4d64066ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 -- 172.21.15.26:0/554865782 >> 172.21.15.26:0/554865782 conn(0x7f4d640717f0 msgr2=0x7f4d640b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:11.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 -- 172.21.15.26:0/554865782 shutdown_connections 2024-01-16T14:55:11.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.089+0000 7f4d6bc27700 1 -- 172.21.15.26:0/554865782 wait complete. 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.090+0000 7f4d6bc27700 1 Processor -- start 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.090+0000 7f4d6bc27700 1 -- start start 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d6406c6b0 0x7f4d641f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d640cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6415bf20 con 0x7f4d6415a600 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641f3760 con 0x7f4d6406c6b0 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6bc27700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64099b90 con 0x7f4d640b3e70 2024-01-16T14:55:11.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6a1c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6a1c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41258/0 (socket says 172.21.15.26:41258) 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/488560579 learned_addr learned my addr 172.21.15.26:0/488560579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d691c2700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d640cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.091+0000 7f4d699c3700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d6406c6b0 0x7f4d641f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 msgr2=0x7f4d640cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d640cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d6406c6b0 msgr2=0x7f4d641f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d6406c6b0 0x7f4d641f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 -- 172.21.15.26:0/488560579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600094a0 con 0x7f4d6415a600 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6a1c4700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406b1b0 tx=0x7f4d5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c012430 con 0x7f4d6415a600 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c020070 con 0x7f4d6415a600 2024-01-16T14:55:11.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c004280 con 0x7f4d6415a600 2024-01-16T14:55:11.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c009a70 con 0x7f4d6415a600 2024-01-16T14:55:11.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.092+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d640adb60 con 0x7f4d6415a600 2024-01-16T14:55:11.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.093+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d64075480 con 0x7f4d6415a600 2024-01-16T14:55:11.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.094+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4d5c0043e0 con 0x7f4d6415a600 2024-01-16T14:55:11.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.094+0000 7f4d5a7fc700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 0x7f4d4806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.094+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d5c022070 con 0x7f4d6415a600 2024-01-16T14:55:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.094+0000 7f4d699c3700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 0x7f4d4806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.095+0000 7f4d699c3700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 0x7f4d4806d810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009d00 tx=0x7f4d60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.099+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d5c0566c0 con 0x7f4d6415a600 2024-01-16T14:55:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.308+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d641596e0 con 0x7f4d6415a600 2024-01-16T14:55:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.309+0000 7f4d5a7fc700 1 -- 172.21.15.26:0/488560579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d5c059ce0 con 0x7f4d6415a600 2024-01-16T14:55:11.293 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 msgr2=0x7f4d4806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 0x7f4d4806d810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009d00 tx=0x7f4d60008040 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 msgr2=0x7f4d640cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d6406b1b0 tx=0x7f4d5c004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 shutdown_connections 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4d4806b350 0x7f4d4806d810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6415a600 0x7f4d640cbc60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4d6406c6b0 0x7f4d641f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 --2- 172.21.15.26:0/488560579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4d640b3e70 0x7f4d640cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 >> 172.21.15.26:0/488560579 conn(0x7f4d640717f0 msgr2=0x7f4d640adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 shutdown_connections 2024-01-16T14:55:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.311+0000 7f4d6bc27700 1 -- 172.21.15.26:0/488560579 wait complete. 2024-01-16T14:55:11.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.465+0000 7f6e3f8f5700 1 Processor -- start 2024-01-16T14:55:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.465+0000 7f6e3f8f5700 1 -- start start 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3f8f5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e380726a0 0x7f6e38072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3f8f5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e3806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3f8f5700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3815f3e0 con 0x7f6e380b3d90 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3f8f5700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380affd0 con 0x7f6e380726a0 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3f8f5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3815e7d0 con 0x7f6e38072fc0 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3d691700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e380726a0 0x7f6e38072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3ce90700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e380b3d90 0x7f6e3806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59552/0 (socket says 172.21.15.26:59552) 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3ce90700 1 -- 172.21.15.26:0/853490892 learned_addr learned my addr 172.21.15.26:0/853490892 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.466+0000 7f6e3de92700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e3806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e3d691700 1 -- 172.21.15.26:0/853490892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 msgr2=0x7f6e3806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e3d691700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e3806c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e3d691700 1 -- 172.21.15.26:0/853490892 >> v1:172.21.15.26:6789/0 conn(0x7f6e380b3d90 legacy=0x7f6e3806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e2ffff700 1 -- 172.21.15.26:0/853490892 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325905545 0 0) 0x7f6e3815f3e0 con 0x7f6e380b3d90 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e3d691700 1 -- 172.21.15.26:0/853490892 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3815eab0 con 0x7f6e380726a0 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e3d691700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380726a0 0x7f6e38072a80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009870 tx=0x7f6e34009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4642191e229a04 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e2ffff700 1 -- 172.21.15.26:0/853490892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34011830 con 0x7f6e380726a0 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e2ffff700 1 -- 172.21.15.26:0/853490892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34011990 con 0x7f6e380726a0 2024-01-16T14:55:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.467+0000 7f6e2ffff700 1 -- 172.21.15.26:0/853490892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34011d20 con 0x7f6e380726a0 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/853490892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380726a0 msgr2=0x7f6e38072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380726a0 0x7f6e38072a80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009870 tx=0x7f6e34009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/853490892 shutdown_connections 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e3806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/853490892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380726a0 0x7f6e38072a80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/853490892 >> 172.21.15.26:0/853490892 conn(0x7f6e380717f0 msgr2=0x7f6e380b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/853490892 shutdown_connections 2024-01-16T14:55:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.468+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/853490892 wait complete. 2024-01-16T14:55:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.469+0000 7f6e3f8f5700 1 Processor -- start 2024-01-16T14:55:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.469+0000 7f6e3f8f5700 1 -- start start 2024-01-16T14:55:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e380726a0 0x7f6e38156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e38158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3809aac0 con 0x7f6e380b3d90 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e381f33c0 con 0x7f6e380726a0 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.470+0000 7f6e3f8f5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e381568d0 con 0x7f6e38072fc0 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41270/0 (socket says 172.21.15.26:41270) 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 -- 172.21.15.26:0/3482689356 learned_addr learned my addr 172.21.15.26:0/3482689356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3ce90700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e38158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3d691700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e380726a0 0x7f6e38156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 -- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 msgr2=0x7f6e38158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e38158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 -- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e380726a0 msgr2=0x7f6e38156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e380726a0 0x7f6e38156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 -- 172.21.15.26:0/3482689356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340094a0 con 0x7f6e380b3d90 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3de92700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e30007e40 tx=0x7f6e3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000ed30 con 0x7f6e380b3d90 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30014420 con 0x7f6e380b3d90 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300147d0 con 0x7f6e380b3d90 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.471+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e38159570 con 0x7f6e380b3d90 2024-01-16T14:55:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.472+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3809a750 con 0x7f6e380b3d90 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.473+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e38159a90 con 0x7f6e380b3d90 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.473+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6e30014930 con 0x7f6e380b3d90 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.474+0000 7f6e2dffb700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 0x7f6e1c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.474+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e3008a240 con 0x7f6e380b3d90 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.474+0000 7f6e3d691700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 0x7f6e1c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.474+0000 7f6e3d691700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 0x7f6e1c06d810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e340090b0 tx=0x7f6e3401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.478+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e300550b0 con 0x7f6e380b3d90 2024-01-16T14:55:11.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.693+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e38075480 con 0x7f6e380b3d90 2024-01-16T14:55:11.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.694+0000 7f6e2dffb700 1 -- 172.21.15.26:0/3482689356 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e300586d0 con 0x7f6e380b3d90 2024-01-16T14:55:11.678 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 msgr2=0x7f6e1c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 0x7f6e1c06d810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e340090b0 tx=0x7f6e3401c040 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 msgr2=0x7f6e3809cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e30007e40 tx=0x7f6e3000e3c0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 shutdown_connections 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6e1c06b350 0x7f6e1c06d810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380b3d90 0x7f6e3809cf80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e380726a0 0x7f6e38156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 --2- 172.21.15.26:0/3482689356 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e38072fc0 0x7f6e38158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 >> 172.21.15.26:0/3482689356 conn(0x7f6e380717f0 msgr2=0x7f6e380ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 shutdown_connections 2024-01-16T14:55:11.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.696+0000 7f6e3f8f5700 1 -- 172.21.15.26:0/3482689356 wait complete. 2024-01-16T14:55:11.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.846+0000 7efc5a19d700 1 Processor -- start 2024-01-16T14:55:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 -- start start 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efc541478c0 0x7efc540b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc5406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5406dbe0 con 0x7efc5406b510 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5406b9a0 con 0x7efc541478c0 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.847+0000 7efc5a19d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc54147ca0 con 0x7efc54161a90 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc53fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc5406b510 0x7efc54071b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59554/0 (socket says 172.21.15.26:59554) 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc53fff700 1 -- 172.21.15.26:0/2104111005 learned_addr learned my addr 172.21.15.26:0/2104111005 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc52ffd700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc5406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc537fe700 1 --2- 172.21.15.26:0/2104111005 >> v2:172.21.15.93:3300/0 conn(0x7efc541478c0 0x7efc540b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047992747 0 0) 0x7efc5406dbe0 con 0x7efc5406b510 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc3c0035b0 con 0x7efc5406b510 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399197449 0 0) 0x7efc3c0035b0 con 0x7efc5406b510 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 msgr2=0x7efc5406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc5406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 >> v2:172.21.15.93:3300/0 conn(0x7efc541478c0 msgr2=0x7efc540b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 --2- 172.21.15.26:0/2104111005 >> v2:172.21.15.93:3300/0 conn(0x7efc541478c0 0x7efc540b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc541622e0 con 0x7efc5406b510 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7efc4c0019f0 con 0x7efc5406b510 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 >> v1:172.21.15.26:6789/0 conn(0x7efc5406b510 legacy=0x7efc54071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc3c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c004d90 0x7efc3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5406b9a0 con 0x7efc3c003fd0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc54147ca0 con 0x7efc3c004d90 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c00ba10 con 0x7efc54161a90 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc52ffd700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc537fe700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc3c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.848+0000 7efc53fff700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c004d90 0x7efc3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 -- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 msgr2=0x7efc3c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc54161a90 0x7efc3c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 -- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c004d90 msgr2=0x7efc3c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc3c004d90 0x7efc3c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 -- 172.21.15.26:0/2104111005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0035b0 con 0x7efc3c003fd0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc52ffd700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc3c004850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc48009b90 tx=0x7efc3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8ca2be0bf15737 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800e6c0 con 0x7efc3c003fd0 2024-01-16T14:55:11.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800e820 con 0x7efc3c003fd0 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.849+0000 7efc527fc700 1 -- 172.21.15.26:0/2104111005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800eb90 con 0x7efc3c003fd0 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 -- 172.21.15.26:0/2104111005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 msgr2=0x7efc3c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 --2- 172.21.15.26:0/2104111005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc3c004850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efc48009b90 tx=0x7efc3c005880 comp rx=0 tx=0).stop 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc53fff700 1 -- 172.21.15.26:0/2104111005 reap_dead start 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 -- 172.21.15.26:0/2104111005 shutdown_connections 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 -- 172.21.15.26:0/2104111005 >> 172.21.15.26:0/2104111005 conn(0x7efc540d0aa0 msgr2=0x7efc540d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 -- 172.21.15.26:0/2104111005 shutdown_connections 2024-01-16T14:55:11.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.850+0000 7efc5a19d700 1 -- 172.21.15.26:0/2104111005 wait complete. 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.851+0000 7efc5a19d700 1 Processor -- start 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 -- start start 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540c7590 0x7efc540b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc540c7a10 0x7efc54071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540c64f0 con 0x7efc3c003fd0 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc540c67d0 con 0x7efc540c7a10 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc5a19d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540c6ab0 con 0x7efc540c7590 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41276/0 (socket says 172.21.15.26:41276) 2024-01-16T14:55:11.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.852+0000 7efc537fe700 1 -- 172.21.15.26:0/3745407476 learned_addr learned my addr 172.21.15.26:0/3745407476 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 -- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540c7590 msgr2=0x7efc540b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540c7590 0x7efc540b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 -- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc540c7a10 msgr2=0x7efc54071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc53fff700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc540c7a10 0x7efc54071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc540c7a10 0x7efc54071b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 -- 172.21.15.26:0/3745407476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540c6d90 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc537fe700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc44011aa0 tx=0x7efc44011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4400f9e0 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4400fb40 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc540c7070 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44014440 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.853+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc541567f0 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.854+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc540c73e0 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.855+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7efc440145a0 con 0x7efc3c003fd0 2024-01-16T14:55:11.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.855+0000 7efc3bfff700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 0x7efc4006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.855+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc4408a400 con 0x7efc3c003fd0 2024-01-16T14:55:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.856+0000 7efc52ffd700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 0x7efc4006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:11.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.856+0000 7efc52ffd700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 0x7efc4006d810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc541571f0 tx=0x7efc48013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:11.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:11.860+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc44055270 con 0x7efc3c003fd0 2024-01-16T14:55:12.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.071+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc54075480 con 0x7efc3c003fd0 2024-01-16T14:55:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.072+0000 7efc3bfff700 1 -- 172.21.15.26:0/3745407476 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7efc44058890 con 0x7efc3c003fd0 2024-01-16T14:55:12.056 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:12.056 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:12.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 msgr2=0x7efc4006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 0x7efc4006d810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc541571f0 tx=0x7efc48013000 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 msgr2=0x7efc541562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efc44011aa0 tx=0x7efc44011e60 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 shutdown_connections 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc540c7a10 0x7efc54071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc4006b350 0x7efc4006d810 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc3c003fd0 0x7efc541562b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 --2- 172.21.15.26:0/3745407476 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540c7590 0x7efc540b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 >> 172.21.15.26:0/3745407476 conn(0x7efc540d0aa0 msgr2=0x7efc541422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 shutdown_connections 2024-01-16T14:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.074+0000 7efc5a19d700 1 -- 172.21.15.26:0/3745407476 wait complete. 2024-01-16T14:55:12.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:12.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:12.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.228+0000 7fd79033c700 1 Processor -- start 2024-01-16T14:55:12.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.228+0000 7fd79033c700 1 -- start start 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.229+0000 7fd79033c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd78806f320 0x7fd78815b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd79033c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd78815c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd79033c700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78806e0d0 con 0x7fd78806d930 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd79033c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78806e3b0 con 0x7fd78806f320 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd79033c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78806e690 con 0x7fd78815c530 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd78806f320 0x7fd78815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e8d9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd78815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd78806f320 0x7fd78815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:34202/0 (socket says 172.21.15.26:34202) 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 -- 172.21.15.26:0/2940268223 learned_addr learned my addr 172.21.15.26:0/2940268223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:12.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78d8d7700 1 --1- 172.21.15.26:0/2940268223 >> v1:172.21.15.93:6789/0 conn(0x7fd78806d930 0x7fd78806f700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:44202/0 (socket says 172.21.15.26:44202) 2024-01-16T14:55:12.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 -- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 msgr2=0x7fd78815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:12.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 --2- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd78815c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 -- 172.21.15.26:0/2940268223 >> v1:172.21.15.93:6789/0 conn(0x7fd78806d930 legacy=0x7fd78806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 -- 172.21.15.26:0/2940268223 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78806e970 con 0x7fd78806f320 2024-01-16T14:55:12.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.230+0000 7fd78e0d8700 1 --2- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806f320 0x7fd78815b160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098f0 tx=0x7fd77c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cbb406da01948ec server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd78d0d6700 1 -- 172.21.15.26:0/2940268223 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00f8a0 con 0x7fd78806f320 2024-01-16T14:55:12.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd78d0d6700 1 -- 172.21.15.26:0/2940268223 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c00fa00 con 0x7fd78806f320 2024-01-16T14:55:12.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd78d0d6700 1 -- 172.21.15.26:0/2940268223 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00fd90 con 0x7fd78806f320 2024-01-16T14:55:12.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 -- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806f320 msgr2=0x7fd78815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 --2- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806f320 0x7fd78815b160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098f0 tx=0x7fd77c009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 -- 172.21.15.26:0/2940268223 shutdown_connections 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 --2- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd78815c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 --2- 172.21.15.26:0/2940268223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806f320 0x7fd78815b160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 -- 172.21.15.26:0/2940268223 >> 172.21.15.26:0/2940268223 conn(0x7fd7880d0b10 msgr2=0x7fd7880d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 -- 172.21.15.26:0/2940268223 shutdown_connections 2024-01-16T14:55:12.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.231+0000 7fd79033c700 1 -- 172.21.15.26:0/2940268223 wait complete. 2024-01-16T14:55:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 Processor -- start 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 -- start start 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806d930 0x7fd7880cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd7881f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880ce470 con 0x7fd78806f320 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7881f4260 con 0x7fd78806d930 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.233+0000 7fd79033c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880ce6a0 con 0x7fd78815c530 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78e0d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806d930 0x7fd7880cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78e8d9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd7881f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41288/0 (socket says 172.21.15.26:41288) 2024-01-16T14:55:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 -- 172.21.15.26:0/861695641 learned_addr learned my addr 172.21.15.26:0/861695641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 -- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 msgr2=0x7fd7881f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd7881f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 -- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806d930 msgr2=0x7fd7880cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806d930 0x7fd7880cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 -- 172.21.15.26:0/861695641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009550 con 0x7fd78806f320 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd78d8d7700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c520 tx=0x7fd78400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784005cb0 con 0x7fd78806f320 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784005e10 con 0x7fd78806f320 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840072d0 con 0x7fd78806f320 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7880ce980 con 0x7fd78806f320 2024-01-16T14:55:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.234+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7881f44b0 con 0x7fd78806f320 2024-01-16T14:55:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.235+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd788070560 con 0x7fd78806f320 2024-01-16T14:55:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.236+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd784007430 con 0x7fd78806f320 2024-01-16T14:55:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.236+0000 7fd77affd700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 0x7fd76c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.236+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd784088db0 con 0x7fd78806f320 2024-01-16T14:55:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.237+0000 7fd78e0d8700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 0x7fd76c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.237+0000 7fd78e0d8700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 0x7fd76c06d810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009c40 tx=0x7fd77c01b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.241+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd784053cd0 con 0x7fd78806f320 2024-01-16T14:55:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.450+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7880ceea0 con 0x7fd78806f320 2024-01-16T14:55:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.451+0000 7fd77affd700 1 -- 172.21.15.26:0/861695641 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7880ceea0 con 0x7fd78806f320 2024-01-16T14:55:12.435 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 msgr2=0x7fd76c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 0x7fd76c06d810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009c40 tx=0x7fd77c01b040 comp rx=0 tx=0).stop 2024-01-16T14:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 msgr2=0x7fd7880cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c520 tx=0x7fd78400c830 comp rx=0 tx=0).stop 2024-01-16T14:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 shutdown_connections 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd78815c530 0x7fd7881f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd78806d930 0x7fd7880cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd76c06b350 0x7fd76c06d810 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 --2- 172.21.15.26:0/861695641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78806f320 0x7fd7880cdf30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 >> 172.21.15.26:0/861695641 conn(0x7fd7880d0b10 msgr2=0x7fd788072650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 shutdown_connections 2024-01-16T14:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.453+0000 7fd79033c700 1 -- 172.21.15.26:0/861695641 wait complete. 2024-01-16T14:55:12.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.608+0000 7f491392b700 1 Processor -- start 2024-01-16T14:55:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.608+0000 7f491392b700 1 -- start start 2024-01-16T14:55:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f491392b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c162870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f491392b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f490c162db0 0x7f490c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f491392b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0b2fb0 con 0x7f490c06cd60 2024-01-16T14:55:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f491392b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490c06d1f0 con 0x7f490c162db0 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f491392b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0b2a30 con 0x7f490c145220 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.609+0000 7f4910ec6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f490c162db0 0x7f490c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f490c162db0 0x7f490c06c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41292/0 (socket says 172.21.15.26:41292) 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 -- 172.21.15.26:0/2362040442 learned_addr learned my addr 172.21.15.26:0/2362040442 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 -- 172.21.15.26:0/2362040442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 msgr2=0x7f490c162870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f49116c7700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c162870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c162870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 -- 172.21.15.26:0/2362040442 >> v1:172.21.15.93:6789/0 conn(0x7f490c06cd60 legacy=0x7f490c147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 -- 172.21.15.26:0/2362040442 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c145600 con 0x7f490c162db0 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f49116c7700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c162870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4910ec6700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c162db0 0x7f490c06c820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098f0 tx=0x7f48fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6077be09a8dd7671 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4903fff700 1 -- 172.21.15.26:0/2362040442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc004230 con 0x7f490c162db0 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4903fff700 1 -- 172.21.15.26:0/2362040442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc004390 con 0x7f490c162db0 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.610+0000 7f4903fff700 1 -- 172.21.15.26:0/2362040442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0036a0 con 0x7f490c162db0 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 -- 172.21.15.26:0/2362040442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c162db0 msgr2=0x7f490c06c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c162db0 0x7f490c06c820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0098f0 tx=0x7f48fc009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 -- 172.21.15.26:0/2362040442 shutdown_connections 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c162870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 --2- 172.21.15.26:0/2362040442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c162db0 0x7f490c06c820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 -- 172.21.15.26:0/2362040442 >> 172.21.15.26:0/2362040442 conn(0x7f490c0b1730 msgr2=0x7f490c0b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 -- 172.21.15.26:0/2362040442 shutdown_connections 2024-01-16T14:55:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.611+0000 7f491392b700 1 -- 172.21.15.26:0/2362040442 wait complete. 2024-01-16T14:55:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.613+0000 7f491392b700 1 Processor -- start 2024-01-16T14:55:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.613+0000 7f491392b700 1 -- start start 2024-01-16T14:55:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c1fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f490c162db0 0x7f490c159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0cf520 con 0x7f490c06cd60 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c158110 con 0x7f490c162db0 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f491392b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c143f20 con 0x7f490c145220 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41294/0 (socket says 172.21.15.26:41294) 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 -- 172.21.15.26:0/2464978003 learned_addr learned my addr 172.21.15.26:0/2464978003 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f4910ec6700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c1fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f4911ec8700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f490c162db0 0x7f490c159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 -- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 msgr2=0x7f490c1fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c1fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 -- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f490c162db0 msgr2=0x7f490c159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f490c162db0 0x7f490c159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.614+0000 7f49116c7700 1 -- 172.21.15.26:0/2464978003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc009550 con 0x7f490c06cd60 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f49116c7700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f490800c750 tx=0x7f490800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908011800 con 0x7f490c06cd60 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908011960 con 0x7f490c06cd60 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908011cd0 con 0x7f490c06cd60 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0ce8a0 con 0x7f490c06cd60 2024-01-16T14:55:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.615+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c0cee60 con 0x7f490c06cd60 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.616+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c075480 con 0x7f490c06cd60 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.616+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4908021350 con 0x7f490c06cd60 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.617+0000 7f4901ffb700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 0x7f48f006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.617+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4908089aa0 con 0x7f490c06cd60 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.617+0000 7f4910ec6700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 0x7f48f006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.618+0000 7f4910ec6700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 0x7f48f006d810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009130 tx=0x7f48fc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.622+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49080549a0 con 0x7f490c06cd60 2024-01-16T14:55:12.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.829+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f490c0b4b40 con 0x7f490c06cd60 2024-01-16T14:55:12.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.830+0000 7f4901ffb700 1 -- 172.21.15.26:0/2464978003 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f490801e020 con 0x7f490c06cd60 2024-01-16T14:55:12.814 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 msgr2=0x7f48f006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 0x7f48f006d810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009130 tx=0x7f48fc008030 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 msgr2=0x7f490c0ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f490800c750 tx=0x7f490800cb10 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 shutdown_connections 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f490c162db0 0x7f490c159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f48f006b350 0x7f48f006d810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c06cd60 0x7f490c0ce290 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 --2- 172.21.15.26:0/2464978003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f490c145220 0x7f490c1fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 >> 172.21.15.26:0/2464978003 conn(0x7f490c0b1730 msgr2=0x7f490c0af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 shutdown_connections 2024-01-16T14:55:12.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.832+0000 7f491392b700 1 -- 172.21.15.26:0/2464978003 wait complete. 2024-01-16T14:55:12.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.987+0000 7fc85168d700 1 Processor -- start 2024-01-16T14:55:12.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.987+0000 7fc85168d700 1 -- start start 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.987+0000 7fc85168d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc84c15e080 0x7fc84c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc85168d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc84c0d2440 0x7fc84c0d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc85168d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c0d1280 con 0x7fc84c0d0740 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc85168d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84c0d14b0 con 0x7fc84c0d2440 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc85168d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84c0d2de0 con 0x7fc84c15e080 2024-01-16T14:55:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc84c0d2440 0x7fc84c0d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc84c0d0740 0x7fc84c0d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59558/0 (socket says 172.21.15.26:59558) 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84b7fe700 1 -- 172.21.15.26:0/3234055403 learned_addr learned my addr 172.21.15.26:0/3234055403 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84affd700 1 --2- 172.21.15.26:0/3234055403 >> v2:172.21.15.93:3300/0 conn(0x7fc84c15e080 0x7fc84c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 -- 172.21.15.26:0/3234055403 >> v2:172.21.15.93:3300/0 conn(0x7fc84c15e080 msgr2=0x7fc84c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3234055403 >> v2:172.21.15.93:3300/0 conn(0x7fc84c15e080 0x7fc84c0cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 -- 172.21.15.26:0/3234055403 >> v1:172.21.15.26:6789/0 conn(0x7fc84c0d0740 legacy=0x7fc84c0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 -- 172.21.15.26:0/3234055403 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c0ce9a0 con 0x7fc84c0d2440 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc849ffb700 1 -- 172.21.15.26:0/3234055403 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321390424 0 0) 0x7fc84c0d1280 con 0x7fc84c0d0740 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3234055403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d2440 0x7fc84c0d28a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b6a80 tx=0x7fc840009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f35086c48ac5a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.988+0000 7fc849ffb700 1 -- 172.21.15.26:0/3234055403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840011830 con 0x7fc84c0d2440 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc849ffb700 1 -- 172.21.15.26:0/3234055403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840011990 con 0x7fc84c0d2440 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc849ffb700 1 -- 172.21.15.26:0/3234055403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840011d20 con 0x7fc84c0d2440 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 -- 172.21.15.26:0/3234055403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d2440 msgr2=0x7fc84c0d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 --2- 172.21.15.26:0/3234055403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d2440 0x7fc84c0d28a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b6a80 tx=0x7fc840009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 -- 172.21.15.26:0/3234055403 shutdown_connections 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 --2- 172.21.15.26:0/3234055403 >> v2:172.21.15.93:3300/0 conn(0x7fc84c15e080 0x7fc84c0cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 --2- 172.21.15.26:0/3234055403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d2440 0x7fc84c0d28a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 -- 172.21.15.26:0/3234055403 >> 172.21.15.26:0/3234055403 conn(0x7fc84c09d180 msgr2=0x7fc84c0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 -- 172.21.15.26:0/3234055403 shutdown_connections 2024-01-16T14:55:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.989+0000 7fc85168d700 1 -- 172.21.15.26:0/3234055403 wait complete. 2024-01-16T14:55:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 Processor -- start 2024-01-16T14:55:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 -- start start 2024-01-16T14:55:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 0x7fc84c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 0x7fc84c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0b6ab0 con 0x7fc84c0d0740 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c159ce0 con 0x7fc84c15e080 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.991+0000 7fc85168d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0c8910 con 0x7fc84c0d2440 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41308/0 (socket says 172.21.15.26:41308) 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 -- 172.21.15.26:0/3075350577 learned_addr learned my addr 172.21.15.26:0/3075350577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 -- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 msgr2=0x7fc84c0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 0x7fc84c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84b7fe700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 0x7fc84c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 0x7fc84c0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 -- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 msgr2=0x7fc84c0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 0x7fc84c0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 -- 172.21.15.26:0/3075350577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400094a0 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 0x7fc84c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84b7fe700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 0x7fc84c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc84affd700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00e870 tx=0x7fc83c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c009590 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c00ee70 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c00ca50 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c0c8bf0 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.992+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c0c91b0 con 0x7fc84c0d0740 2024-01-16T14:55:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.993+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c0b6c20 con 0x7fc84c0d0740 2024-01-16T14:55:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.994+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc83c00cbb0 con 0x7fc84c0d0740 2024-01-16T14:55:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.994+0000 7fc8337fe700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 0x7fc83806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.994+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc83c089b10 con 0x7fc84c0d0740 2024-01-16T14:55:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.995+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 0x7fc83806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.995+0000 7fc84a7fc700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 0x7fc83806d810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b87d0 tx=0x7fc840008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:12.999+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc83c054980 con 0x7fc84c0d0740 2024-01-16T14:55:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.210+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc84c075480 con 0x7fc84c0d0740 2024-01-16T14:55:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.211+0000 7fc8337fe700 1 -- 172.21.15.26:0/3075350577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc83c0210a0 con 0x7fc84c0d0740 2024-01-16T14:55:13.195 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 msgr2=0x7fc83806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 0x7fc83806d810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0b87d0 tx=0x7fc840008030 comp rx=0 tx=0).stop 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 msgr2=0x7fc84c1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00e870 tx=0x7fc83c00ec30 comp rx=0 tx=0).stop 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 shutdown_connections 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc84c15e080 0x7fc84c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc83806b350 0x7fc83806d810 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c0d0740 0x7fc84c1597a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 --2- 172.21.15.26:0/3075350577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc84c0d2440 0x7fc84c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 >> 172.21.15.26:0/3075350577 conn(0x7fc84c09d180 msgr2=0x7fc84c15d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 shutdown_connections 2024-01-16T14:55:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.213+0000 7fc85168d700 1 -- 172.21.15.26:0/3075350577 wait complete. 2024-01-16T14:55:13.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:13.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.363+0000 7ff1b0e47700 1 Processor -- start 2024-01-16T14:55:13.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.363+0000 7ff1b0e47700 1 -- start start 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1b0e47700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1ac06afd0 0x7ff1ac06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1b0e47700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 0x7ff1ac06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1b0e47700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ac162240 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1b0e47700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac06b8f0 con 0x7ff1ac06afd0 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1b0e47700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac06e810 con 0x7ff1ac1626d0 2024-01-16T14:55:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1ac06afd0 0x7ff1ac06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aad9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1ac06e2b0 0x7ff1ac071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59568/0 (socket says 172.21.15.26:59568) 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aad9d700 1 -- 172.21.15.26:0/2848406573 learned_addr learned my addr 172.21.15.26:0/2848406573 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1a9d9b700 1 --2- 172.21.15.26:0/2848406573 >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 0x7ff1ac06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 -- 172.21.15.26:0/2848406573 >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 msgr2=0x7ff1ac06dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 --2- 172.21.15.26:0/2848406573 >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 0x7ff1ac06dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 -- 172.21.15.26:0/2848406573 >> v1:172.21.15.26:6789/0 conn(0x7ff1ac06e2b0 legacy=0x7ff1ac071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 -- 172.21.15.26:0/2848406573 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac161bf0 con 0x7ff1ac06afd0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1a959a700 1 -- 172.21.15.26:0/2848406573 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596806977 0 0) 0x7ff1ac162240 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1a9d9b700 1 --2- 172.21.15.26:0/2848406573 >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 0x7ff1ac06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.364+0000 7ff1aa59c700 1 --2- 172.21.15.26:0/2848406573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06afd0 0x7ff1ac06b3b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0b5b70 tx=0x7ff19c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cd621d9a9527bf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1a959a700 1 -- 172.21.15.26:0/2848406573 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c011830 con 0x7ff1ac06afd0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1a959a700 1 -- 172.21.15.26:0/2848406573 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c011990 con 0x7ff1ac06afd0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1a959a700 1 -- 172.21.15.26:0/2848406573 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c011d20 con 0x7ff1ac06afd0 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 -- 172.21.15.26:0/2848406573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06afd0 msgr2=0x7ff1ac06b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/2848406573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06afd0 0x7ff1ac06b3b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0b5b70 tx=0x7ff19c009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 -- 172.21.15.26:0/2848406573 shutdown_connections 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/2848406573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06afd0 0x7ff1ac06b3b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/2848406573 >> v2:172.21.15.93:3300/0 conn(0x7ff1ac1626d0 0x7ff1ac06dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 -- 172.21.15.26:0/2848406573 >> 172.21.15.26:0/2848406573 conn(0x7ff1ac0b2740 msgr2=0x7ff1ac0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 -- 172.21.15.26:0/2848406573 shutdown_connections 2024-01-16T14:55:13.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.365+0000 7ff1b0e47700 1 -- 172.21.15.26:0/2848406573 wait complete. 2024-01-16T14:55:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.367+0000 7ff1b0e47700 1 Processor -- start 2024-01-16T14:55:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.367+0000 7ff1b0e47700 1 -- start start 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1ac06afd0 0x7ff1ac142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1ac1626d0 0x7ff1ac1400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac151820 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac150a50 con 0x7ff1ac1626d0 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1b0e47700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1430c0 con 0x7ff1ac06afd0 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41312/0 (socket says 172.21.15.26:41312) 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 -- 172.21.15.26:0/1767174585 learned_addr learned my addr 172.21.15.26:0/1767174585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1aa59c700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1ac06afd0 0x7ff1ac142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1aad9d700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1ac1626d0 0x7ff1ac1400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 -- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1ac06afd0 msgr2=0x7ff1ac142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1ac06afd0 0x7ff1ac142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 -- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1ac1626d0 msgr2=0x7ff1ac1400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1ac1626d0 0x7ff1ac1400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 -- 172.21.15.26:0/1767174585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0094a0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.368+0000 7ff1a9d9b700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c820 tx=0x7ff19400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.369+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194011800 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.369+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194011960 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.369+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac152cb0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.369+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194011cf0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.369+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac1515f0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.370+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ac075480 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.370+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff1940233a0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.371+0000 7ff1a2ffd700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 0x7ff18c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.371+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff194089e40 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.371+0000 7ff1aa59c700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 0x7ff18c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.371+0000 7ff1aa59c700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 0x7ff18c06d810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0090b0 tx=0x7ff19c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.375+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff194054c20 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.591+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1ac0702f0 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.592+0000 7ff1a2ffd700 1 -- 172.21.15.26:0/1767174585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7ff19401f070 con 0x7ff1ac06e2b0 2024-01-16T14:55:13.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:13.577 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.594+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 msgr2=0x7ff18c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.594+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 0x7ff18c06d810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0090b0 tx=0x7ff19c008030 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 msgr2=0x7ff1ac13f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c820 tx=0x7ff19400cb30 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 shutdown_connections 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1ac1626d0 0x7ff1ac1400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1ac06afd0 0x7ff1ac142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff18c06b350 0x7ff18c06d810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 --2- 172.21.15.26:0/1767174585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ac06e2b0 0x7ff1ac13f750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 >> 172.21.15.26:0/1767174585 conn(0x7ff1ac0b2740 msgr2=0x7ff1ac15b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 shutdown_connections 2024-01-16T14:55:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.595+0000 7ff1b0e47700 1 -- 172.21.15.26:0/1767174585 wait complete. 2024-01-16T14:55:13.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:13.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.748+0000 7f1f919fd700 1 Processor -- start 2024-01-16T14:55:13.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.748+0000 7f1f919fd700 1 -- start start 2024-01-16T14:55:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f919fd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f919fd700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0ce9a0 con 0x7f1f8c15e080 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f919fd700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c0cec40 con 0x7f1f8c0d0740 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f919fd700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c0d14b0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f8a7fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f8affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f8c15e080 0x7f1f8c0d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59572/0 (socket says 172.21.15.26:59572) 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f8affd700 1 -- 172.21.15.26:0/2513431996 learned_addr learned my addr 172.21.15.26:0/2513431996 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.749+0000 7f1f8a7fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41324/0 (socket says 172.21.15.26:41324) 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/2513431996 >> v1:172.21.15.93:6789/0 conn(0x7f1f8c0d0740 legacy=0x7f1f8c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/2513431996 >> v1:172.21.15.26:6789/0 conn(0x7f1f8c15e080 legacy=0x7f1f8c0d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/2513431996 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0cef20 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f89ffb700 1 -- 172.21.15.26:0/2513431996 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082082531 0 0) 0x7f1f8c0ce9a0 con 0x7f1f8c15e080 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f8a7fc700 1 --2- 172.21.15.26:0/2513431996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009840 tx=0x7f1f80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fccdcac0636e86 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f89ffb700 1 -- 172.21.15.26:0/2513431996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011830 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f89ffb700 1 -- 172.21.15.26:0/2513431996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011990 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f89ffb700 1 -- 172.21.15.26:0/2513431996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011d20 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.750+0000 7f1f919fd700 1 -- 172.21.15.26:0/2513431996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 msgr2=0x7f1f8c0cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 --2- 172.21.15.26:0/2513431996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009840 tx=0x7f1f80009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 -- 172.21.15.26:0/2513431996 shutdown_connections 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 --2- 172.21.15.26:0/2513431996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c0cfa00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 -- 172.21.15.26:0/2513431996 >> 172.21.15.26:0/2513431996 conn(0x7f1f8c09d180 msgr2=0x7f1f8c0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 -- 172.21.15.26:0/2513431996 shutdown_connections 2024-01-16T14:55:13.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.751+0000 7f1f919fd700 1 -- 172.21.15.26:0/2513431996 wait complete. 2024-01-16T14:55:13.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.752+0000 7f1f919fd700 1 Processor -- start 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.752+0000 7f1f919fd700 1 -- start start 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f8c0d0740 0x7f1f8c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f8c15e080 0x7f1f8c0b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c14e8a0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0b87a0 con 0x7f1f8c15e080 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f919fd700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0b8a80 con 0x7f1f8c0d0740 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41326/0 (socket says 172.21.15.26:41326) 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/1902819768 learned_addr learned my addr 172.21.15.26:0/1902819768 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8affd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f8c0d0740 0x7f1f8c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8b7fe700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f8c15e080 0x7f1f8c0b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f8c0d0740 msgr2=0x7f1f8c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f8c0d0740 0x7f1f8c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f8c15e080 msgr2=0x7f1f8c0b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f8c15e080 0x7f1f8c0b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.753+0000 7f1f8a7fc700 1 -- 172.21.15.26:0/1902819768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800094a0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f8a7fc700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f800090b0 tx=0x7f1f80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001a7e0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8001a940 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001acb0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c0b8d60 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.754+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c1595a0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.755+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c0b9280 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.755+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1f80022070 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.756+0000 7f1f73fff700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 0x7f1f7806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.756+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f8008a9b0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.756+0000 7f1f8affd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 0x7f1f7806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:13.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.757+0000 7f1f8affd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 0x7f1f7806d810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c007c40 tx=0x7f1f7c007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.761+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f800559f0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.970+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f8c075480 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.971+0000 7f1f73fff700 1 -- 172.21.15.26:0/1902819768 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f800150a0 con 0x7f1f8c0d2b50 2024-01-16T14:55:13.955 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:13.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 msgr2=0x7f1f7806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 0x7f1f7806d810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c007c40 tx=0x7f1f7c007b90 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 msgr2=0x7f1f8c14e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f800090b0 tx=0x7f1f80004750 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 shutdown_connections 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f8c15e080 0x7f1f8c0b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f8c0d0740 0x7f1f8c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1f7806b350 0x7f1f7806d810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 --2- 172.21.15.26:0/1902819768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f8c0d2b50 0x7f1f8c14e360 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.973+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 >> 172.21.15.26:0/1902819768 conn(0x7f1f8c09d180 msgr2=0x7f1f8c0d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.974+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 shutdown_connections 2024-01-16T14:55:13.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:13.974+0000 7f1f919fd700 1 -- 172.21.15.26:0/1902819768 wait complete. 2024-01-16T14:55:13.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:14.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.126+0000 7f12abead700 1 Processor -- start 2024-01-16T14:55:14.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.127+0000 7f12abead700 1 -- start start 2024-01-16T14:55:14.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.127+0000 7f12abead700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 0x7f12a40ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.127+0000 7f12abead700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40ae470 con 0x7f12a40b0490 2024-01-16T14:55:14.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.127+0000 7f12abead700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a40ad860 con 0x7f12a406bae0 2024-01-16T14:55:14.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.127+0000 7f12abead700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a40adb40 con 0x7f12a415a570 2024-01-16T14:55:14.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12aa44a700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12a40b0490 0x7f12a40ae900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59578/0 (socket says 172.21.15.26:59578) 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12aa44a700 1 -- 172.21.15.26:0/954423202 learned_addr learned my addr 172.21.15.26:0/954423202 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a9c49700 1 --2- 172.21.15.26:0/954423202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 0x7f12a40ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179857017 0 0) 0x7f12a40ae470 con 0x7f12a40b0490 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12900035b0 con 0x7f12a40b0490 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096801363 0 0) 0x7f12a40ad860 con 0x7f12a406bae0 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a9c49700 1 -- 172.21.15.26:0/954423202 >> v1:172.21.15.93:6789/0 conn(0x7f12a406bae0 legacy=0x7f12a40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a9c49700 1 -- 172.21.15.26:0/954423202 >> v1:172.21.15.26:6789/0 conn(0x7f12a40b0490 legacy=0x7f12a40ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a9c49700 1 -- 172.21.15.26:0/954423202 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40ade20 con 0x7f12a415a570 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.128+0000 7f12a9c49700 1 --2- 172.21.15.26:0/954423202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 0x7f12a40ad1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009870 tx=0x7f12a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92786fd1f843d6d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0010800 con 0x7f12a415a570 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0010960 con 0x7f12a415a570 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12a8c47700 1 -- 172.21.15.26:0/954423202 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0010cf0 con 0x7f12a415a570 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 -- 172.21.15.26:0/954423202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 msgr2=0x7f12a40ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 --2- 172.21.15.26:0/954423202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 0x7f12a40ad1f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009870 tx=0x7f12a0009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:14.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 -- 172.21.15.26:0/954423202 shutdown_connections 2024-01-16T14:55:14.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 --2- 172.21.15.26:0/954423202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a415a570 0x7f12a40ad1f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 -- 172.21.15.26:0/954423202 >> 172.21.15.26:0/954423202 conn(0x7f12a40717f0 msgr2=0x7f12a40b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:14.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 -- 172.21.15.26:0/954423202 shutdown_connections 2024-01-16T14:55:14.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.129+0000 7f12abead700 1 -- 172.21.15.26:0/954423202 wait complete. 2024-01-16T14:55:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 Processor -- start 2024-01-16T14:55:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 -- start start 2024-01-16T14:55:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 0x7f12a409ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 0x7f12a415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a41f3710 con 0x7f12a406bae0 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a4155d20 con 0x7f12a415a570 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.131+0000 7f12abead700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a409b6d0 con 0x7f12a40b0490 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41328/0 (socket says 172.21.15.26:41328) 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 -- 172.21.15.26:0/2843148889 learned_addr learned my addr 172.21.15.26:0/2843148889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12aa44a700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 0x7f12a415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9448700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 0x7f12a409ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 -- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 msgr2=0x7f12a409ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 0x7f12a409ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 -- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 msgr2=0x7f12a415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 0x7f12a415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 -- 172.21.15.26:0/2843148889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00094a0 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9448700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 0x7f12a409ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12aa44a700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 0x7f12a415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12a9c49700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009bc0 tx=0x7f12a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0018ba0 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0018d00 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a409b9b0 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0022630 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.132+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a409bf70 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.133+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4075480 con 0x7f12a406bae0 2024-01-16T14:55:14.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.134+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f12a0021390 con 0x7f12a406bae0 2024-01-16T14:55:14.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.134+0000 7f129a7fc700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 0x7f128806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.134+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12a008ac40 con 0x7f12a406bae0 2024-01-16T14:55:14.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.135+0000 7f12a9448700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 0x7f128806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.135+0000 7f12a9448700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 0x7f128806d810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12a406c760 tx=0x7f1294009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.139+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12a0055ac0 con 0x7f12a406bae0 2024-01-16T14:55:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.348+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12a41f3850 con 0x7f12a406bae0 2024-01-16T14:55:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.349+0000 7f129a7fc700 1 -- 172.21.15.26:0/2843148889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f12a0009300 con 0x7f12a406bae0 2024-01-16T14:55:14.333 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 msgr2=0x7f128806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 0x7f128806d810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12a406c760 tx=0x7f1294009040 comp rx=0 tx=0).stop 2024-01-16T14:55:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 msgr2=0x7f12a41557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009bc0 tx=0x7f12a00047c0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 shutdown_connections 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a415a570 0x7f12a415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f128806b350 0x7f128806d810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a406bae0 0x7f12a41557e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 --2- 172.21.15.26:0/2843148889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a40b0490 0x7f12a409ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 >> 172.21.15.26:0/2843148889 conn(0x7f12a40717f0 msgr2=0x7f12a415d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 shutdown_connections 2024-01-16T14:55:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.351+0000 7f12abead700 1 -- 172.21.15.26:0/2843148889 wait complete. 2024-01-16T14:55:14.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.504+0000 7f7e637e7700 1 Processor -- start 2024-01-16T14:55:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.504+0000 7f7e637e7700 1 -- start start 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.505+0000 7f7e637e7700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0726a0 0x7f7e5c072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.505+0000 7f7e637e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.505+0000 7f7e637e7700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c15f3e0 con 0x7f7e5c0b3d90 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.505+0000 7f7e637e7700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c0affd0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.505+0000 7f7e637e7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c15e7d0 con 0x7f7e5c072fc0 2024-01-16T14:55:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61583700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0726a0 0x7f7e5c072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:45526/0 (socket says 172.21.15.26:45526) 2024-01-16T14:55:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 -- 172.21.15.26:0/3525701631 learned_addr learned my addr 172.21.15.26:0/3525701631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 -- 172.21.15.26:0/3525701631 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0726a0 msgr2=0x7f7e5c072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 --2- 172.21.15.26:0/3525701631 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0726a0 0x7f7e5c072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 -- 172.21.15.26:0/3525701631 >> v1:172.21.15.93:6789/0 conn(0x7f7e5c0b3d90 legacy=0x7f7e5c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 -- 172.21.15.26:0/3525701631 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c15eab0 con 0x7f7e5c072fc0 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e61d84700 1 --2- 172.21.15.26:0/3525701631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009e80 tx=0x7f7e5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450c65dbcdb6115f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e53fff700 1 -- 172.21.15.26:0/3525701631 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400e070 con 0x7f7e5c072fc0 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e53fff700 1 -- 172.21.15.26:0/3525701631 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5400b8d0 con 0x7f7e5c072fc0 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.506+0000 7f7e53fff700 1 -- 172.21.15.26:0/3525701631 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400bc80 con 0x7f7e5c072fc0 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 -- 172.21.15.26:0/3525701631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 msgr2=0x7f7e5c06c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 --2- 172.21.15.26:0/3525701631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009e80 tx=0x7f7e5400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 -- 172.21.15.26:0/3525701631 shutdown_connections 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 --2- 172.21.15.26:0/3525701631 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c072fc0 0x7f7e5c06c1d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 --2- 172.21.15.26:0/3525701631 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0726a0 0x7f7e5c072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 -- 172.21.15.26:0/3525701631 >> 172.21.15.26:0/3525701631 conn(0x7f7e5c0717f0 msgr2=0x7f7e5c0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 -- 172.21.15.26:0/3525701631 shutdown_connections 2024-01-16T14:55:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.507+0000 7f7e637e7700 1 -- 172.21.15.26:0/3525701631 wait complete. 2024-01-16T14:55:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.508+0000 7f7e637e7700 1 Processor -- start 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.508+0000 7f7e637e7700 1 -- start start 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c072fc0 0x7f7e5c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c0b3d90 0x7f7e5c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c1544e0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c1547c0 con 0x7f7e5c072fc0 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e637e7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c158800 con 0x7f7e5c0b3d90 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e61583700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e61583700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58462/0 (socket says 172.21.15.26:58462) 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.509+0000 7f7e61583700 1 -- 172.21.15.26:0/1186801226 learned_addr learned my addr 172.21.15.26:0/1186801226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61d84700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c0b3d90 0x7f7e5c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e60d82700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c072fc0 0x7f7e5c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 -- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c0b3d90 msgr2=0x7f7e5c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c0b3d90 0x7f7e5c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 -- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c072fc0 msgr2=0x7f7e5c1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c072fc0 0x7f7e5c1595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 -- 172.21.15.26:0/1186801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54009b30 con 0x7f7e5c0726a0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e61583700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e8a0 tx=0x7f7e5800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58009590 con 0x7f7e5c0726a0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58013680 con 0x7f7e5c0726a0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580139f0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c158ae0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.510+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c147b40 con 0x7f7e5c0726a0 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.511+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c06c9a0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.512+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7e58013b50 con 0x7f7e5c0726a0 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.512+0000 7f7e51ffb700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 0x7f7e4006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.512+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e58089aa0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.513+0000 7f7e60d82700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 0x7f7e4006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.513+0000 7f7e60d82700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 0x7f7e4006d810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c1f7630 tx=0x7f7e4c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.517+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e58054870 con 0x7f7e5c0726a0 2024-01-16T14:55:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.727+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e5c075480 con 0x7f7e5c0726a0 2024-01-16T14:55:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.728+0000 7f7e51ffb700 1 -- 172.21.15.26:0/1186801226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e580210a0 con 0x7f7e5c0726a0 2024-01-16T14:55:14.712 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 msgr2=0x7f7e4006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 0x7f7e4006d810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c1f7630 tx=0x7f7e4c009040 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 msgr2=0x7f7e5c147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e8a0 tx=0x7f7e5800ebb0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 shutdown_connections 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c0b3d90 0x7f7e5c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f7e4006b350 0x7f7e4006d810 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e5c0726a0 0x7f7e5c147600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.730+0000 7f7e637e7700 1 --2- 172.21.15.26:0/1186801226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c072fc0 0x7f7e5c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.731+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 >> 172.21.15.26:0/1186801226 conn(0x7f7e5c0717f0 msgr2=0x7f7e5c15faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.731+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 shutdown_connections 2024-01-16T14:55:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.731+0000 7f7e637e7700 1 -- 172.21.15.26:0/1186801226 wait complete. 2024-01-16T14:55:14.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.880+0000 7f695ce29700 1 Processor -- start 2024-01-16T14:55:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.880+0000 7f695ce29700 1 -- start start 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.881+0000 7f695ce29700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69580d0ae0 0x7f695806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.881+0000 7f695ce29700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f69580d0f60 0x7f695806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.881+0000 7f695ce29700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69580cfd60 con 0x7f695815e000 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.881+0000 7f695ce29700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69580cf4b0 con 0x7f69580d0ae0 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.881+0000 7f695ce29700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69580cf1d0 con 0x7f69580d0f60 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69580d0ae0 0x7f695806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695659c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f695815e000 0x7f695806eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42162/0 (socket says 172.21.15.26:42162) 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695659c700 1 -- 172.21.15.26:0/2182076526 learned_addr learned my addr 172.21.15.26:0/2182076526 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 --2- 172.21.15.26:0/2182076526 >> v2:172.21.15.26:3300/0 conn(0x7f69580d0ae0 0x7f695806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58472/0 (socket says 172.21.15.26:58472) 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6956d9d700 1 --2- 172.21.15.26:0/2182076526 >> v2:172.21.15.93:3300/0 conn(0x7f69580d0f60 0x7f695806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 -- 172.21.15.26:0/2182076526 >> v2:172.21.15.93:3300/0 conn(0x7f69580d0f60 msgr2=0x7f695806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 --2- 172.21.15.26:0/2182076526 >> v2:172.21.15.93:3300/0 conn(0x7f69580d0f60 0x7f695806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 -- 172.21.15.26:0/2182076526 >> v1:172.21.15.26:6789/0 conn(0x7f695815e000 legacy=0x7f695806eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695559a700 1 -- 172.21.15.26:0/2182076526 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756524369 0 0) 0x7f69580cfd60 con 0x7f695815e000 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 -- 172.21.15.26:0/2182076526 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580d0000 con 0x7f69580d0ae0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f6955d9b700 1 --2- 172.21.15.26:0/2182076526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69580d0ae0 0x7f695806f6f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6958072210 tx=0x7f694c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458825734485fa29 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695559a700 1 -- 172.21.15.26:0/2182076526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c011830 con 0x7f69580d0ae0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695559a700 1 -- 172.21.15.26:0/2182076526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c011990 con 0x7f69580d0ae0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.882+0000 7f695559a700 1 -- 172.21.15.26:0/2182076526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c011d20 con 0x7f69580d0ae0 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 -- 172.21.15.26:0/2182076526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69580d0ae0 msgr2=0x7f695806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 --2- 172.21.15.26:0/2182076526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69580d0ae0 0x7f695806f6f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6958072210 tx=0x7f694c009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 -- 172.21.15.26:0/2182076526 shutdown_connections 2024-01-16T14:55:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 --2- 172.21.15.26:0/2182076526 >> v2:172.21.15.93:3300/0 conn(0x7f69580d0f60 0x7f695806fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 --2- 172.21.15.26:0/2182076526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69580d0ae0 0x7f695806f6f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 -- 172.21.15.26:0/2182076526 >> 172.21.15.26:0/2182076526 conn(0x7f69580b3720 msgr2=0x7f69580b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 -- 172.21.15.26:0/2182076526 shutdown_connections 2024-01-16T14:55:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.883+0000 7f695ce29700 1 -- 172.21.15.26:0/2182076526 wait complete. 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.885+0000 7f695ce29700 1 Processor -- start 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.885+0000 7f695ce29700 1 -- start start 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.885+0000 7f695ce29700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f69580d0ae0 0x7f69580b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.885+0000 7f695ce29700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69580d0f60 0x7f69580afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695ce29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695ce29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c6840 con 0x7f695815e000 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695ce29700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c5bc0 con 0x7f69580d0f60 2024-01-16T14:55:14.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695ce29700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c5ea0 con 0x7f69580d0ae0 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58478/0 (socket says 172.21.15.26:58478) 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 -- 172.21.15.26:0/4185052357 learned_addr learned my addr 172.21.15.26:0/4185052357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695659c700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f69580d0ae0 0x7f69580b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6955d9b700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69580d0f60 0x7f69580afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 -- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f69580d0ae0 msgr2=0x7f69580b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f69580d0ae0 0x7f69580b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 -- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69580d0f60 msgr2=0x7f69580afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69580d0f60 0x7f69580afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 -- 172.21.15.26:0/4185052357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c0094a0 con 0x7f695815e000 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6956d9d700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6950007e60 tx=0x7f695000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000ede0 con 0x7f695815e000 2024-01-16T14:55:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950014420 con 0x7f695815e000 2024-01-16T14:55:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69580c6180 con 0x7f695815e000 2024-01-16T14:55:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.886+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500147d0 con 0x7f695815e000 2024-01-16T14:55:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.887+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69580b5dc0 con 0x7f695815e000 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.888+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69580b43f0 con 0x7f695815e000 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.888+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6950014930 con 0x7f695815e000 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.889+0000 7f6946ffd700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 0x7f693806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.889+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695008a300 con 0x7f695815e000 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.889+0000 7f695659c700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 0x7f693806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.889+0000 7f695659c700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 0x7f693806d810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6948000f60 tx=0x7f6948007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:14.894+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6950055170 con 0x7f695815e000 2024-01-16T14:55:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.106+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6958075480 con 0x7f695815e000 2024-01-16T14:55:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.107+0000 7f6946ffd700 1 -- 172.21.15.26:0/4185052357 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f6950058790 con 0x7f695815e000 2024-01-16T14:55:15.091 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:15.091 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 msgr2=0x7f693806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 0x7f693806d810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6948000f60 tx=0x7f6948007c00 comp rx=0 tx=0).stop 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 msgr2=0x7f69580b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6950007e60 tx=0x7f695000e3c0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 shutdown_connections 2024-01-16T14:55:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f693806b350 0x7f693806d810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f695815e000 0x7f69580b0190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f69580d0ae0 0x7f69580b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 --2- 172.21.15.26:0/4185052357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69580d0f60 0x7f69580afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 >> 172.21.15.26:0/4185052357 conn(0x7f69580b3720 msgr2=0x7f69580b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 shutdown_connections 2024-01-16T14:55:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.109+0000 7f695ce29700 1 -- 172.21.15.26:0/4185052357 wait complete. 2024-01-16T14:55:15.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:15.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:15.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.263+0000 7f49931a3700 1 Processor -- start 2024-01-16T14:55:15.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.264+0000 7f49931a3700 1 -- start start 2024-01-16T14:55:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.264+0000 7f49931a3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c06f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.264+0000 7f49931a3700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 0x7f498c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.264+0000 7f49931a3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c15e990 con 0x7f498c06f9e0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f49931a3700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c15ec70 con 0x7f498c0722c0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f49931a3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15ef50 con 0x7f498c158730 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f498bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f498c06f9e0 0x7f498c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42176/0 (socket says 172.21.15.26:42176) 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f498bfff700 1 -- 172.21.15.26:0/3838014717 learned_addr learned my addr 172.21.15.26:0/3838014717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3838014717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c06f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4991740700 1 --2- 172.21.15.26:0/3838014717 >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 0x7f498c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f498b7fe700 1 -- 172.21.15.26:0/3838014717 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779287669 0 0) 0x7f498c15e990 con 0x7f498c06f9e0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f498b7fe700 1 -- 172.21.15.26:0/3838014717 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f497c0035b0 con 0x7f498c06f9e0 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 -- 172.21.15.26:0/3838014717 >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 msgr2=0x7f498c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:15.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3838014717 >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 0x7f498c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 -- 172.21.15.26:0/3838014717 >> v1:172.21.15.26:6789/0 conn(0x7f498c06f9e0 legacy=0x7f498c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 -- 172.21.15.26:0/3838014717 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c15f230 con 0x7f498c158730 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4991740700 1 --2- 172.21.15.26:0/3838014717 >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 0x7f498c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.265+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3838014717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c06f4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31083ee46491d125 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f498b7fe700 1 -- 172.21.15.26:0/3838014717 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011830 con 0x7f498c158730 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f498b7fe700 1 -- 172.21.15.26:0/3838014717 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980011990 con 0x7f498c158730 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f498b7fe700 1 -- 172.21.15.26:0/3838014717 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980011d20 con 0x7f498c158730 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 -- 172.21.15.26:0/3838014717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 msgr2=0x7f498c06f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 --2- 172.21.15.26:0/3838014717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c06f4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4980009840 tx=0x7f4980009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 -- 172.21.15.26:0/3838014717 shutdown_connections 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 --2- 172.21.15.26:0/3838014717 >> v2:172.21.15.93:3300/0 conn(0x7f498c0722c0 0x7f498c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 --2- 172.21.15.26:0/3838014717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c06f4a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 -- 172.21.15.26:0/3838014717 >> 172.21.15.26:0/3838014717 conn(0x7f498c0b27b0 msgr2=0x7f498c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 -- 172.21.15.26:0/3838014717 shutdown_connections 2024-01-16T14:55:15.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.266+0000 7f49931a3700 1 -- 172.21.15.26:0/3838014717 wait complete. 2024-01-16T14:55:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.268+0000 7f49931a3700 1 Processor -- start 2024-01-16T14:55:15.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.268+0000 7f49931a3700 1 -- start start 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 0x7f498c143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15d8e0 con 0x7f498c06f9e0 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15dbc0 con 0x7f498c0722c0 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49931a3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c156050 con 0x7f498c158730 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58492/0 (socket says 172.21.15.26:58492) 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 -- 172.21.15.26:0/3304605852 learned_addr learned my addr 172.21.15.26:0/3304605852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4991740700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f498bfff700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 0x7f498c143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 -- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 msgr2=0x7f498c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 -- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 msgr2=0x7f498c143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 0x7f498c143e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 -- 172.21.15.26:0/3304605852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984009a70 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4991740700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f498bfff700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 0x7f498c143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f4990f3f700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4980009fd0 tx=0x7f4980004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800036a0 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498001e070 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.269+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980003a60 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.270+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49800094a0 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.270+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c156610 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.271+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c075480 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.271+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4980003bc0 con 0x7f498c06f9e0 2024-01-16T14:55:15.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.271+0000 7f49897fa700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 0x7f497006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.272+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f498008aa90 con 0x7f498c06f9e0 2024-01-16T14:55:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.272+0000 7f498bfff700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 0x7f497006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.272+0000 7f498bfff700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 0x7f497006d810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f49780011e0 tx=0x7f4978009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.276+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4980055990 con 0x7f498c06f9e0 2024-01-16T14:55:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.485+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f498c166970 con 0x7f498c06f9e0 2024-01-16T14:55:15.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.486+0000 7f49897fa700 1 -- 172.21.15.26:0/3304605852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49800150a0 con 0x7f498c06f9e0 2024-01-16T14:55:15.470 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 msgr2=0x7f497006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 0x7f497006d810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f49780011e0 tx=0x7f4978009040 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 msgr2=0x7f498c145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4980009fd0 tx=0x7f4980004750 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 shutdown_connections 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f498c158730 0x7f498c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f497006b350 0x7f497006d810 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c06f9e0 0x7f498c145b70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 --2- 172.21.15.26:0/3304605852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f498c0722c0 0x7f498c143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 >> 172.21.15.26:0/3304605852 conn(0x7f498c0b27b0 msgr2=0x7f498c0d0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.488+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 shutdown_connections 2024-01-16T14:55:15.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.489+0000 7f49931a3700 1 -- 172.21.15.26:0/3304605852 wait complete. 2024-01-16T14:55:15.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.642+0000 7f41ff5ea700 1 Processor -- start 2024-01-16T14:55:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 -- start start 2024-01-16T14:55:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 0x7f41f815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f80cc670 con 0x7f41f815c1c0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f80cffb0 con 0x7f41f80b4240 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.643+0000 7f41ff5ea700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80cf7a0 con 0x7f41f815bde0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fdb87700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 0x7f41f815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fcb85700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f41f815c1c0 0x7f41f80cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:34008/0 (socket says 172.21.15.26:34008) 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:45568/0 (socket says 172.21.15.26:45568) 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fcb85700 1 -- 172.21.15.26:0/3771672324 learned_addr learned my addr 172.21.15.26:0/3771672324 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 -- 172.21.15.26:0/3771672324 >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 msgr2=0x7f41f815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 --2- 172.21.15.26:0/3771672324 >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 0x7f41f815fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 -- 172.21.15.26:0/3771672324 >> v1:172.21.15.93:6789/0 conn(0x7f41f815c1c0 legacy=0x7f41f80cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 -- 172.21.15.26:0/3771672324 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80cf9f0 con 0x7f41f815bde0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fdb87700 1 --2- 172.21.15.26:0/3771672324 >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 0x7f41f815fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41ebfff700 1 -- 172.21.15.26:0/3771672324 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358352560 0 0) 0x7f41f80cc670 con 0x7f41f815c1c0 2024-01-16T14:55:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.644+0000 7f41fd386700 1 --2- 172.21.15.26:0/3771672324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009840 tx=0x7f41ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef93d0dfe4c3ea8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ebfff700 1 -- 172.21.15.26:0/3771672324 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011830 con 0x7f41f815bde0 2024-01-16T14:55:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ebfff700 1 -- 172.21.15.26:0/3771672324 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec011990 con 0x7f41f815bde0 2024-01-16T14:55:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ebfff700 1 -- 172.21.15.26:0/3771672324 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec011d20 con 0x7f41f815bde0 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 -- 172.21.15.26:0/3771672324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 msgr2=0x7f41f806ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/3771672324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009840 tx=0x7f41ec009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 -- 172.21.15.26:0/3771672324 shutdown_connections 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/3771672324 >> v2:172.21.15.93:3300/0 conn(0x7f41f80b4240 0x7f41f815fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/3771672324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f815bde0 0x7f41f806ece0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 -- 172.21.15.26:0/3771672324 >> 172.21.15.26:0/3771672324 conn(0x7f41f80b3720 msgr2=0x7f41f80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 -- 172.21.15.26:0/3771672324 shutdown_connections 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.645+0000 7f41ff5ea700 1 -- 172.21.15.26:0/3771672324 wait complete. 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.647+0000 7f41ff5ea700 1 Processor -- start 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.647+0000 7f41ff5ea700 1 -- start start 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.647+0000 7f41ff5ea700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f80b4240 0x7f41f815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.647+0000 7f41ff5ea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41f815bde0 0x7f41f8152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41ff5ea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41ff5ea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f815ebd0 con 0x7f41f80cdff0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41ff5ea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80ce9c0 con 0x7f41f815bde0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41ff5ea700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f8153170 con 0x7f41f80b4240 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58498/0 (socket says 172.21.15.26:58498) 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 -- 172.21.15.26:0/2841751779 learned_addr learned my addr 172.21.15.26:0/2841751779 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fd386700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f80b4240 0x7f41f815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fcb85700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41f815bde0 0x7f41f8152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 -- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f80b4240 msgr2=0x7f41f815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f80b4240 0x7f41f815eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 -- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41f815bde0 msgr2=0x7f41f8152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41f815bde0 0x7f41f8152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 -- 172.21.15.26:0/2841751779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009a70 con 0x7f41f80cdff0 2024-01-16T14:55:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41fdb87700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41f000efc0 tx=0x7f41f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000ca90 con 0x7f41f80cdff0 2024-01-16T14:55:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f000f040 con 0x7f41f80cdff0 2024-01-16T14:55:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.648+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec0094a0 con 0x7f41f80cdff0 2024-01-16T14:55:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.649+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000ce30 con 0x7f41f80cdff0 2024-01-16T14:55:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.649+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f815df50 con 0x7f41f80cdff0 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.650+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f815ed10 con 0x7f41f80cdff0 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.650+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f41f0014480 con 0x7f41f80cdff0 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.650+0000 7f41e9ffb700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 0x7f41dc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.651+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f008a210 con 0x7f41f80cdff0 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.651+0000 7f41fd386700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 0x7f41dc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.651+0000 7f41fd386700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 0x7f41dc06d810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0090b0 tx=0x7f41ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.655+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41f0055080 con 0x7f41f80cdff0 2024-01-16T14:55:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.863+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41f806ece0 con 0x7f41f80cdff0 2024-01-16T14:55:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.863+0000 7f41e9ffb700 1 -- 172.21.15.26:0/2841751779 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41f00586a0 con 0x7f41f80cdff0 2024-01-16T14:55:15.847 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:15.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 msgr2=0x7f41dc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 0x7f41dc06d810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0090b0 tx=0x7f41ec008030 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 msgr2=0x7f41f80ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f41f000efc0 tx=0x7f41f000c3a0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 shutdown_connections 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f41dc06b350 0x7f41dc06d810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f80cdff0 0x7f41f80ce480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41f80b4240 0x7f41f815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 --2- 172.21.15.26:0/2841751779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41f815bde0 0x7f41f8152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 >> 172.21.15.26:0/2841751779 conn(0x7f41f80b3720 msgr2=0x7f41f80b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 shutdown_connections 2024-01-16T14:55:15.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:15.865+0000 7f41ff5ea700 1 -- 172.21.15.26:0/2841751779 wait complete. 2024-01-16T14:55:15.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.019+0000 7f3e9fa4c700 1 Processor -- start 2024-01-16T14:55:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.019+0000 7f3e9fa4c700 1 -- start start 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9fa4c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e98071b20 0x7f3e9806b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9fa4c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806f4a0 0x7f3e98071f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9fa4c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e980cffd0 con 0x7f3e9806ea50 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9fa4c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e98072980 con 0x7f3e98071b20 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9fa4c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9806f9b0 con 0x7f3e9806f4a0 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9d7e8700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e98071b20 0x7f3e9806b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9dfe9700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e9806ea50 0x7f3e9815f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42192/0 (socket says 172.21.15.26:42192) 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.020+0000 7f3e9dfe9700 1 -- 172.21.15.26:0/4180814954 learned_addr learned my addr 172.21.15.26:0/4180814954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9cfe7700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806f4a0 0x7f3e98071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9d7e8700 1 -- 172.21.15.26:0/4180814954 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806f4a0 msgr2=0x7f3e98071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806f4a0 0x7f3e98071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9d7e8700 1 -- 172.21.15.26:0/4180814954 >> v1:172.21.15.26:6789/0 conn(0x7f3e9806ea50 legacy=0x7f3e9815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9d7e8700 1 -- 172.21.15.26:0/4180814954 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9806bed0 con 0x7f3e98071b20 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e8ffff700 1 -- 172.21.15.26:0/4180814954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958432151 0 0) 0x7f3e90001490 con 0x7f3e9806ea50 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9806b990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009840 tx=0x7f3e94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd80811009e91f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e8ffff700 1 -- 172.21.15.26:0/4180814954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94011830 con 0x7f3e98071b20 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e8ffff700 1 -- 172.21.15.26:0/4180814954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94011990 con 0x7f3e98071b20 2024-01-16T14:55:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.021+0000 7f3e8ffff700 1 -- 172.21.15.26:0/4180814954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94011d20 con 0x7f3e98071b20 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/4180814954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 msgr2=0x7f3e9806b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9806b990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009840 tx=0x7f3e94009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/4180814954 shutdown_connections 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9806b990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/4180814954 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806f4a0 0x7f3e98071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/4180814954 >> 172.21.15.26:0/4180814954 conn(0x7f3e9806c0e0 msgr2=0x7f3e9806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/4180814954 shutdown_connections 2024-01-16T14:55:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.022+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/4180814954 wait complete. 2024-01-16T14:55:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.023+0000 7f3e9fa4c700 1 Processor -- start 2024-01-16T14:55:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.023+0000 7f3e9fa4c700 1 -- start start 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 0x7f3e9812db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9806f4a0 0x7f3e98154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9813c970 con 0x7f3e98071b20 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9813cc50 con 0x7f3e9806f4a0 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9fa4c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9813cf30 con 0x7f3e9806ea50 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9dfe9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9dfe9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58522/0 (socket says 172.21.15.26:58522) 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9dfe9700 1 -- 172.21.15.26:0/714170882 learned_addr learned my addr 172.21.15.26:0/714170882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 0x7f3e9812db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.024+0000 7f3e9cfe7700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9806f4a0 0x7f3e98154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 -- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 msgr2=0x7f3e9812db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 0x7f3e9812db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 -- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9806f4a0 msgr2=0x7f3e98154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9806f4a0 0x7f3e98154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 -- 172.21.15.26:0/714170882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940094a0 con 0x7f3e98071b20 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 0x7f3e9812db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9dfe9700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e90002230 tx=0x7f3e9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000abb0 con 0x7f3e98071b20 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000ad30 con 0x7f3e98071b20 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9813d210 con 0x7f3e98071b20 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000aea0 con 0x7f3e98071b20 2024-01-16T14:55:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.025+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e98155520 con 0x7f3e98071b20 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.027+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e98075480 con 0x7f3e98071b20 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.027+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3e90007310 con 0x7f3e98071b20 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.027+0000 7f3e8dffb700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 0x7f3e7c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.027+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e90089850 con 0x7f3e98071b20 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.028+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 0x7f3e7c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.028+0000 7f3e9d7e8700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 0x7f3e7c06d810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e94000c00 tx=0x7f3e9401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.032+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e900546c0 con 0x7f3e98071b20 2024-01-16T14:55:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.241+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e980cffd0 con 0x7f3e98071b20 2024-01-16T14:55:16.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.241+0000 7f3e8dffb700 1 -- 172.21.15.26:0/714170882 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e980cffd0 con 0x7f3e98071b20 2024-01-16T14:55:16.225 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.243+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 msgr2=0x7f3e7c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.243+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 0x7f3e7c06d810 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3e94000c00 tx=0x7f3e9401c040 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 msgr2=0x7f3e9815e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e90002230 tx=0x7f3e9000cf90 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 shutdown_connections 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f3e7c06b350 0x7f3e7c06d810 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e98071b20 0x7f3e9815e1d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9806ea50 0x7f3e9812db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 --2- 172.21.15.26:0/714170882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9806f4a0 0x7f3e98154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 >> 172.21.15.26:0/714170882 conn(0x7f3e9806c0e0 msgr2=0x7f3e980cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 shutdown_connections 2024-01-16T14:55:16.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.244+0000 7f3e9fa4c700 1 -- 172.21.15.26:0/714170882 wait complete. 2024-01-16T14:55:16.309 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) Test that when one clone is in progress, one progress bar is printed 2024-01-16T14:55:16.309 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-16T14:55:16.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:16.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.465+0000 7f2e3f594700 1 Processor -- start 2024-01-16T14:55:16.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.465+0000 7f2e3f594700 1 -- start start 2024-01-16T14:55:16.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3f594700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e380d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3f594700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e380b3370 con 0x7f2e380cf480 2024-01-16T14:55:16.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3f594700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e380cfc40 con 0x7f2e380d2ba0 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3f594700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b35c0 con 0x7f2e3815e460 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3cb2f700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2e380cf480 0x7f2e380d2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42194/0 (socket says 172.21.15.26:42194) 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3cb2f700 1 -- 172.21.15.26:0/2533954612 learned_addr learned my addr 172.21.15.26:0/2533954612 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e380d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873519632 0 0) 0x7f2e380b3370 con 0x7f2e380cf480 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.466+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e240035b0 con 0x7f2e380cf480 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724465656 0 0) 0x7f2e380cfc40 con 0x7f2e380d2ba0 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e380b3370 con 0x7f2e380d2ba0 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1087572812 0 0) 0x7f2e240035b0 con 0x7f2e380cf480 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 msgr2=0x7f2e380d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e380d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 >> v1:172.21.15.93:6789/0 conn(0x7f2e380d2ba0 legacy=0x7f2e380b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380b38a0 con 0x7f2e380cf480 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f2e2c001670 con 0x7f2e380cf480 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e380d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 >> v1:172.21.15.26:6789/0 conn(0x7f2e380cf480 legacy=0x7f2e380d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e24003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e24004d90 0x7f2e24005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380b35c0 con 0x7f2e24003bc0 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2400bac0 con 0x7f2e24004d90 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2400bc80 con 0x7f2e3815e460 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e24003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3cb2f700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3db31700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e24004d90 0x7f2e24005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 -- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 msgr2=0x7f2e24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:16.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 -- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e24004d90 msgr2=0x7f2e24005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e24004d90 0x7f2e24005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3d330700 1 -- 172.21.15.26:0/2533954612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340094a0 con 0x7f2e24003bc0 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.467+0000 7f2e3cb2f700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e3815e460 0x7f2e24004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3d330700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e24003fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009d30 tx=0x7f2e24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698b29e238aa75d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34009e80 con 0x7f2e24003bc0 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3400e9a0 con 0x7f2e24003bc0 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e2bfff700 1 -- 172.21.15.26:0/2533954612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400ed10 con 0x7f2e24003bc0 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 -- 172.21.15.26:0/2533954612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 msgr2=0x7f2e24003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 --2- 172.21.15.26:0/2533954612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e24003fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009d30 tx=0x7f2e24005880 comp rx=0 tx=0).stop 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3db31700 1 -- 172.21.15.26:0/2533954612 reap_dead start 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 -- 172.21.15.26:0/2533954612 shutdown_connections 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 -- 172.21.15.26:0/2533954612 >> 172.21.15.26:0/2533954612 conn(0x7f2e380d3eb0 msgr2=0x7f2e380d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 -- 172.21.15.26:0/2533954612 shutdown_connections 2024-01-16T14:55:16.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.468+0000 7f2e3f594700 1 -- 172.21.15.26:0/2533954612 wait complete. 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.470+0000 7f2e3f594700 1 Processor -- start 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.470+0000 7f2e3f594700 1 -- start start 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e380cbbb0 0x7f2e380cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3815bd00 0x7f2e3815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380cddc0 con 0x7f2e24003bc0 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38152900 con 0x7f2e3815bd00 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3f594700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3815c6c0 con 0x7f2e380cbbb0 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3d330700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3d330700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58538/0 (socket says 172.21.15.26:58538) 2024-01-16T14:55:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3d330700 1 -- 172.21.15.26:0/1489860970 learned_addr learned my addr 172.21.15.26:0/1489860970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.471+0000 7f2e3cb2f700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e380cbbb0 0x7f2e380cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3db31700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3815bd00 0x7f2e3815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 -- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e380cbbb0 msgr2=0x7f2e380cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e380cbbb0 0x7f2e380cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 -- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3815bd00 msgr2=0x7f2e3815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3815bd00 0x7f2e3815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 -- 172.21.15.26:0/1489860970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e380ce1e0 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3d330700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005810 tx=0x7f2e340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34003ab0 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34003c30 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e380cd140 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34009e80 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.472+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e380cd700 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.473+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e38075480 con 0x7f2e24003bc0 2024-01-16T14:55:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.474+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2e34013470 con 0x7f2e24003bc0 2024-01-16T14:55:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.474+0000 7f2e29ffb700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 0x7f2e1c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.475+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e3408b6d0 con 0x7f2e24003bc0 2024-01-16T14:55:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.475+0000 7f2e3cb2f700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 0x7f2e1c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.475+0000 7f2e3cb2f700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 0x7f2e1c06d810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e3815bcb0 tx=0x7f2e2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.479+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e34056650 con 0x7f2e24003bc0 2024-01-16T14:55:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.692+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e380d2270 con 0x7f2e24003bc0 2024-01-16T14:55:16.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.694+0000 7f2e29ffb700 1 -- 172.21.15.26:0/1489860970 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f2e34059c70 con 0x7f2e24003bc0 2024-01-16T14:55:16.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:16.678 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 msgr2=0x7f2e1c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 0x7f2e1c06d810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e3815bcb0 tx=0x7f2e2c009040 comp rx=0 tx=0).stop 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 msgr2=0x7f2e381523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e24005810 tx=0x7f2e340047c0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 shutdown_connections 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e3815bd00 0x7f2e3815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2e1c06b350 0x7f2e1c06d810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24003bc0 0x7f2e381523c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 --2- 172.21.15.26:0/1489860970 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e380cbbb0 0x7f2e380cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 >> 172.21.15.26:0/1489860970 conn(0x7f2e380d3eb0 msgr2=0x7f2e380d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 shutdown_connections 2024-01-16T14:55:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.696+0000 7f2e3f594700 1 -- 172.21.15.26:0/1489860970 wait complete. 2024-01-16T14:55:16.694 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:16.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.850+0000 7f4e4d95a700 1 Processor -- start 2024-01-16T14:55:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.850+0000 7f4e4d95a700 1 -- start start 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e4d95a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e4815e200 0x7f4e4806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e4d95a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4e4806c6e0 0x7f4e480d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e4d95a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e480d12b0 con 0x7f4e480b3270 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e4d95a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480d0980 con 0x7f4e4815e200 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e4d95a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480d0c60 con 0x7f4e4806c6e0 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e46ffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e4815e200 0x7f4e4806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e467fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4e480b3270 0x7f4e4806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42196/0 (socket says 172.21.15.26:42196) 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e467fc700 1 -- 172.21.15.26:0/2482608978 learned_addr learned my addr 172.21.15.26:0/2482608978 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.851+0000 7f4e477fe700 1 --2- 172.21.15.26:0/2482608978 >> v2:172.21.15.93:3300/0 conn(0x7f4e4806c6e0 0x7f4e480d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e46ffd700 1 -- 172.21.15.26:0/2482608978 >> v2:172.21.15.93:3300/0 conn(0x7f4e4806c6e0 msgr2=0x7f4e480d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e46ffd700 1 --2- 172.21.15.26:0/2482608978 >> v2:172.21.15.93:3300/0 conn(0x7f4e4806c6e0 0x7f4e480d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e46ffd700 1 -- 172.21.15.26:0/2482608978 >> v1:172.21.15.26:6789/0 conn(0x7f4e480b3270 legacy=0x7f4e4806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e45ffb700 1 -- 172.21.15.26:0/2482608978 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818274408 0 0) 0x7f4e480d12b0 con 0x7f4e480b3270 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e46ffd700 1 -- 172.21.15.26:0/2482608978 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480d0f40 con 0x7f4e4815e200 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e46ffd700 1 --2- 172.21.15.26:0/2482608978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e4806b9a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009840 tx=0x7f4e38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb8c87d3205af0f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e45ffb700 1 -- 172.21.15.26:0/2482608978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011830 con 0x7f4e4815e200 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e45ffb700 1 -- 172.21.15.26:0/2482608978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38011990 con 0x7f4e4815e200 2024-01-16T14:55:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.852+0000 7f4e45ffb700 1 -- 172.21.15.26:0/2482608978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38011d20 con 0x7f4e4815e200 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 -- 172.21.15.26:0/2482608978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 msgr2=0x7f4e4806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/2482608978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e4806b9a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009840 tx=0x7f4e38009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 -- 172.21.15.26:0/2482608978 shutdown_connections 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/2482608978 >> v2:172.21.15.93:3300/0 conn(0x7f4e4806c6e0 0x7f4e480d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/2482608978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e4806b9a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 -- 172.21.15.26:0/2482608978 >> 172.21.15.26:0/2482608978 conn(0x7f4e4809d180 msgr2=0x7f4e480d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 -- 172.21.15.26:0/2482608978 shutdown_connections 2024-01-16T14:55:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.853+0000 7f4e4d95a700 1 -- 172.21.15.26:0/2482608978 wait complete. 2024-01-16T14:55:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.854+0000 7f4e4d95a700 1 Processor -- start 2024-01-16T14:55:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.855+0000 7f4e4d95a700 1 -- start start 2024-01-16T14:55:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.855+0000 7f4e4d95a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e4d95a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e4d95a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e481f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e4d95a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480b4690 con 0x7f4e4815e200 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e4d95a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480b87a0 con 0x7f4e4806c6e0 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e4d95a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48157bb0 con 0x7f4e480b3270 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:34436/0 (socket says 172.21.15.26:34436) 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:45646/0 (socket says 172.21.15.26:45646) 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 -- 172.21.15.26:0/3126814407 learned_addr learned my addr 172.21.15.26:0/3126814407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e477fe700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e481f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 -- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 msgr2=0x7f4e480b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 -- 172.21.15.26:0/3126814407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 msgr2=0x7f4e481f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e481f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 -- 172.21.15.26:0/3126814407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380094a0 con 0x7f4e4806c6e0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e467fc700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e477fe700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e481f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.856+0000 7f4e46ffd700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e38013040 tx=0x7f4e38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.857+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380036a0 con 0x7f4e4806c6e0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.857+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3801e070 con 0x7f4e4806c6e0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.857+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e480b7b90 con 0x7f4e4806c6e0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.857+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38003a20 con 0x7f4e4806c6e0 2024-01-16T14:55:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.857+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e480b8150 con 0x7f4e4806c6e0 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.858+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e480b8a80 con 0x7f4e4806c6e0 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.858+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4e38003b80 con 0x7f4e4806c6e0 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.859+0000 7f4e2ffff700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 0x7f4e3406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.859+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e3808a9c0 con 0x7f4e4806c6e0 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.859+0000 7f4e467fc700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 0x7f4e3406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.859+0000 7f4e467fc700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 0x7f4e3406d810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b4900 tx=0x7f4e3c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:16.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:16.864+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e380558a0 con 0x7f4e4806c6e0 2024-01-16T14:55:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.073+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e480b2f60 con 0x7f4e4806c6e0 2024-01-16T14:55:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.075+0000 7f4e2ffff700 1 -- 172.21.15.26:0/3126814407 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e38020020 con 0x7f4e4806c6e0 2024-01-16T14:55:17.058 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.076+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 msgr2=0x7f4e3406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.076+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 0x7f4e3406d810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b4900 tx=0x7f4e3c007de0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.076+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 msgr2=0x7f4e48157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.076+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e38013040 tx=0x7f4e38004750 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 shutdown_connections 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4e3406b350 0x7f4e3406d810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e4815e200 0x7f4e481f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e4806c6e0 0x7f4e48157670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 --2- 172.21.15.26:0/3126814407 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e480b3270 0x7f4e480b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 >> 172.21.15.26:0/3126814407 conn(0x7f4e4809d180 msgr2=0x7f4e4815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 shutdown_connections 2024-01-16T14:55:17.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.077+0000 7f4e4d95a700 1 -- 172.21.15.26:0/3126814407 wait complete. 2024-01-16T14:55:17.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:17.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.230+0000 7fc405f7a700 1 Processor -- start 2024-01-16T14:55:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.230+0000 7fc405f7a700 1 -- start start 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc405f7a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4000726a0 0x7fc400072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc405f7a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc400072fc0 0x7fc40006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc405f7a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40015f3e0 con 0x7fc4000b3d90 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc405f7a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4000affd0 con 0x7fc4000726a0 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc405f7a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40015e7d0 con 0x7fc400072fc0 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc3ff7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4000726a0 0x7fc400072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc3feffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc4000b3d90 0x7fc40006bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42200/0 (socket says 172.21.15.26:42200) 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.231+0000 7fc3feffd700 1 -- 172.21.15.26:0/723472624 learned_addr learned my addr 172.21.15.26:0/723472624 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3fffff700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc400072fc0 0x7fc40006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3ff7fe700 1 -- 172.21.15.26:0/723472624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc400072fc0 msgr2=0x7fc40006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc400072fc0 0x7fc40006c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3ff7fe700 1 -- 172.21.15.26:0/723472624 >> v1:172.21.15.26:6789/0 conn(0x7fc4000b3d90 legacy=0x7fc40006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3ff7fe700 1 -- 172.21.15.26:0/723472624 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40015eab0 con 0x7fc4000726a0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4000726a0 0x7fc400072a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009870 tx=0x7fc3f0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feedd0a2e2858fbf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3fe7fc700 1 -- 172.21.15.26:0/723472624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0011830 con 0x7fc4000726a0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3fe7fc700 1 -- 172.21.15.26:0/723472624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f0011990 con 0x7fc4000726a0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.232+0000 7fc3fe7fc700 1 -- 172.21.15.26:0/723472624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0011d20 con 0x7fc4000726a0 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 -- 172.21.15.26:0/723472624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4000726a0 msgr2=0x7fc400072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4000726a0 0x7fc400072a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009870 tx=0x7fc3f0009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 -- 172.21.15.26:0/723472624 shutdown_connections 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc400072fc0 0x7fc40006c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 --2- 172.21.15.26:0/723472624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4000726a0 0x7fc400072a80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 -- 172.21.15.26:0/723472624 >> 172.21.15.26:0/723472624 conn(0x7fc4000717f0 msgr2=0x7fc4000b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 -- 172.21.15.26:0/723472624 shutdown_connections 2024-01-16T14:55:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.233+0000 7fc405f7a700 1 -- 172.21.15.26:0/723472624 wait complete. 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.235+0000 7fc405f7a700 1 Processor -- start 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.235+0000 7fc405f7a700 1 -- start start 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 0x7fc40009ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc4000b3d90 0x7fc40015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40015a090 con 0x7fc400072fc0 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40015a370 con 0x7fc4000726a0 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc405f7a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40015a650 con 0x7fc4000b3d90 2024-01-16T14:55:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58566/0 (socket says 172.21.15.26:58566) 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 -- 172.21.15.26:0/1338256396 learned_addr learned my addr 172.21.15.26:0/1338256396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 0x7fc40009ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3fffff700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc4000b3d90 0x7fc40015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 -- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc4000b3d90 msgr2=0x7fc40015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc4000b3d90 0x7fc40015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 -- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 msgr2=0x7fc40009ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 0x7fc40009ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 -- 172.21.15.26:0/1338256396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00094a0 con 0x7fc400072fc0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 0x7fc40009ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.236+0000 7fc3feffd700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c520 tx=0x7fc3e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.237+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005d70 con 0x7fc400072fc0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.237+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e80070d0 con 0x7fc400072fc0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.237+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007440 con 0x7fc400072fc0 2024-01-16T14:55:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.237+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40015a930 con 0x7fc400072fc0 2024-01-16T14:55:17.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.237+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40014fcf0 con 0x7fc400072fc0 2024-01-16T14:55:17.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.238+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40015ae50 con 0x7fc400072fc0 2024-01-16T14:55:17.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.238+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc3e80075a0 con 0x7fc400072fc0 2024-01-16T14:55:17.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.239+0000 7fc3f7fff700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 0x7fc3e006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.239+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3e8088f10 con 0x7fc400072fc0 2024-01-16T14:55:17.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.239+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 0x7fc3e006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.239+0000 7fc3ff7fe700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 0x7fc3e006d810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009790 tx=0x7fc3f001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.243+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc40015ae50 con 0x7fc400072fc0 2024-01-16T14:55:17.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.458+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc40015ae50 con 0x7fc400072fc0 2024-01-16T14:55:17.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.459+0000 7fc3f7fff700 1 -- 172.21.15.26:0/1338256396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3e8017020 con 0x7fc400072fc0 2024-01-16T14:55:17.443 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 msgr2=0x7fc3e006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 0x7fc3e006d810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009790 tx=0x7fc3f001c040 comp rx=0 tx=0).stop 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 msgr2=0x7fc400148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c520 tx=0x7fc3e800c8e0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 shutdown_connections 2024-01-16T14:55:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc4000b3d90 0x7fc40015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4000726a0 0x7fc40009ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3e006b350 0x7fc3e006d810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 --2- 172.21.15.26:0/1338256396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc400072fc0 0x7fc400148570 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 >> 172.21.15.26:0/1338256396 conn(0x7fc4000717f0 msgr2=0x7fc4000ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 shutdown_connections 2024-01-16T14:55:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.461+0000 7fc405f7a700 1 -- 172.21.15.26:0/1338256396 wait complete. 2024-01-16T14:55:17.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:17.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.615+0000 7fe8b1b4c700 1 Processor -- start 2024-01-16T14:55:17.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.615+0000 7fe8b1b4c700 1 -- start start 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8b1b4c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac15be40 0x7fe8ac0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8b1b4c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8b1b4c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0af940 con 0x7fe8ac0b1a90 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8b1b4c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac0b1130 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8b1b4c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac067420 con 0x7fe8ac15be40 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8aaffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8abfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe8ac0b1a90 0x7fe8ac06b7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42212/0 (socket says 172.21.15.26:42212) 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8abfff700 1 -- 172.21.15.26:0/1744565184 learned_addr learned my addr 172.21.15.26:0/1744565184 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/1744565184 >> v2:172.21.15.26:3300/0 conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58576/0 (socket says 172.21.15.26:58576) 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.616+0000 7fe8ab7fe700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac15be40 0x7fe8ac0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aaffd700 1 -- 172.21.15.26:0/1744565184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac15be40 msgr2=0x7fe8ac0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac15be40 0x7fe8ac0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aaffd700 1 -- 172.21.15.26:0/1744565184 >> v1:172.21.15.26:6789/0 conn(0x7fe8ac0b1a90 legacy=0x7fe8ac06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aa7fc700 1 -- 172.21.15.26:0/1744565184 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146859779 0 0) 0x7fe8a4001490 con 0x7fe8ac0b1a90 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aaffd700 1 -- 172.21.15.26:0/1744565184 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0b0ab0 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009870 tx=0x7fe8a0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7975a2c84e79fdf server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aa7fc700 1 -- 172.21.15.26:0/1744565184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011830 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aa7fc700 1 -- 172.21.15.26:0/1744565184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0011990 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.617+0000 7fe8aa7fc700 1 -- 172.21.15.26:0/1744565184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011d20 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/1744565184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0d0ec0 msgr2=0x7fe8ac0b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009870 tx=0x7fe8a0009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:17.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/1744565184 shutdown_connections 2024-01-16T14:55:17.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac15be40 0x7fe8ac0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/1744565184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac0b1550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/1744565184 >> 172.21.15.26:0/1744565184 conn(0x7fe8ac0717f0 msgr2=0x7fe8ac0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/1744565184 shutdown_connections 2024-01-16T14:55:17.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.618+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/1744565184 wait complete. 2024-01-16T14:55:17.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.619+0000 7fe8b1b4c700 1 Processor -- start 2024-01-16T14:55:17.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.619+0000 7fe8b1b4c700 1 -- start start 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac0ad1f0 0x7fe8ac0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0cc090 con 0x7fe8ac15be40 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ae370 con 0x7fe8ac0d0ec0 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8b1b4c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0ad760 con 0x7fe8ac0ad1f0 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8aaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8aaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58582/0 (socket says 172.21.15.26:58582) 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.620+0000 7fe8aaffd700 1 -- 172.21.15.26:0/558703682 learned_addr learned my addr 172.21.15.26:0/558703682 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8ab7fe700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8abfff700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac0ad1f0 0x7fe8ac0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 -- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac0ad1f0 msgr2=0x7fe8ac0ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac0ad1f0 0x7fe8ac0ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 -- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ac0d0ec0 msgr2=0x7fe8ac1f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac1f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 -- 172.21.15.26:0/558703682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a00094a0 con 0x7fe8ac15be40 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8aaffd700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00036a0 con 0x7fe8ac15be40 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a001f070 con 0x7fe8ac15be40 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac0ada40 con 0x7fe8ac15be40 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0003a00 con 0x7fe8ac15be40 2024-01-16T14:55:17.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.621+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0ae000 con 0x7fe8ac15be40 2024-01-16T14:55:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.622+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac075480 con 0x7fe8ac15be40 2024-01-16T14:55:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.623+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe8a0003b60 con 0x7fe8ac15be40 2024-01-16T14:55:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.623+0000 7fe893fff700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 0x7fe89806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.623+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a0021070 con 0x7fe8ac15be40 2024-01-16T14:55:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.623+0000 7fe8ab7fe700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 0x7fe89806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.624+0000 7fe8ab7fe700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 0x7fe89806d810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f7110 tx=0x7fe89c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.628+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8a00563a0 con 0x7fe8ac15be40 2024-01-16T14:55:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.838+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8ac0ae4b0 con 0x7fe8ac15be40 2024-01-16T14:55:17.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.838+0000 7fe893fff700 1 -- 172.21.15.26:0/558703682 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8a0015020 con 0x7fe8ac15be40 2024-01-16T14:55:17.822 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 msgr2=0x7fe89806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 0x7fe89806d810 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac1f7110 tx=0x7fe89c008040 comp rx=0 tx=0).stop 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 msgr2=0x7fe8ac0cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0004750 comp rx=0 tx=0).stop 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 shutdown_connections 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ac0ad1f0 0x7fe8ac0ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ac0d0ec0 0x7fe8ac1f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe89806b350 0x7fe89806d810 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 --2- 172.21.15.26:0/558703682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ac15be40 0x7fe8ac0cbb50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 >> 172.21.15.26:0/558703682 conn(0x7fe8ac0717f0 msgr2=0x7fe8ac0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 shutdown_connections 2024-01-16T14:55:17.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.841+0000 7fe8b1b4c700 1 -- 172.21.15.26:0/558703682 wait complete. 2024-01-16T14:55:17.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-16T14:55:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.992+0000 7fcb3b5e7700 1 Processor -- start 2024-01-16T14:55:17.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 -- start start 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb340b57b0 0x7fcb3406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 0x7fcb3406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3406dbe0 con 0x7fcb341478c0 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3406b9a0 con 0x7fcb340b57b0 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.993+0000 7fcb3b5e7700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb34147ca0 con 0x7fcb3406b510 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb340b57b0 0x7fcb3406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb340b57b0 0x7fcb3406afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58598/0 (socket says 172.21.15.26:58598) 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 -- 172.21.15.26:0/1186086789 learned_addr learned my addr 172.21.15.26:0/1186086789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb39b84700 1 --2- 172.21.15.26:0/1186086789 >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 0x7fcb3406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 -- 172.21.15.26:0/1186086789 >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 msgr2=0x7fcb3406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 --2- 172.21.15.26:0/1186086789 >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 0x7fcb3406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 -- 172.21.15.26:0/1186086789 >> v1:172.21.15.93:6789/0 conn(0x7fcb341478c0 legacy=0x7fcb34161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 -- 172.21.15.26:0/1186086789 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb341622e0 con 0x7fcb340b57b0 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb39b84700 1 --2- 172.21.15.26:0/1186086789 >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 0x7fcb3406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb38b82700 1 --2- 172.21.15.26:0/1186086789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3406afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406c040 tx=0x7fcb30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e52c72e7fdbb5e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb27fff700 1 -- 172.21.15.26:0/1186086789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004230 con 0x7fcb340b57b0 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb27fff700 1 -- 172.21.15.26:0/1186086789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30004390 con 0x7fcb340b57b0 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.994+0000 7fcb27fff700 1 -- 172.21.15.26:0/1186086789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300036a0 con 0x7fcb340b57b0 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/1186086789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 msgr2=0x7fcb3406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/1186086789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3406afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb3406c040 tx=0x7fcb30009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/1186086789 shutdown_connections 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/1186086789 >> v2:172.21.15.93:3300/0 conn(0x7fcb3406b510 0x7fcb3406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/1186086789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3406afd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/1186086789 >> 172.21.15.26:0/1186086789 conn(0x7fcb340d0aa0 msgr2=0x7fcb340d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/1186086789 shutdown_connections 2024-01-16T14:55:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.995+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/1186086789 wait complete. 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.996+0000 7fcb3b5e7700 1 Processor -- start 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.996+0000 7fcb3b5e7700 1 -- start start 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 0x7fcb34155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 0x7fcb341466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34155a00 con 0x7fcb340b57b0 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340adce0 con 0x7fcb3406b510 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb3b5e7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340b14f0 con 0x7fcb341478c0 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb38b82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb38b82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58604/0 (socket says 172.21.15.26:58604) 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.997+0000 7fcb38b82700 1 -- 172.21.15.26:0/3335406089 learned_addr learned my addr 172.21.15.26:0/3335406089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb39b84700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 0x7fcb341466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb39383700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 0x7fcb34155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 -- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 msgr2=0x7fcb341466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 0x7fcb341466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 -- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 msgr2=0x7fcb34155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 0x7fcb34155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 -- 172.21.15.26:0/3335406089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30009550 con 0x7fcb340b57b0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb39b84700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 0x7fcb341466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb39383700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 0x7fcb34155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb38b82700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb340717b0 tx=0x7fcb30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30003d80 con 0x7fcb340b57b0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3001f070 con 0x7fcb340b57b0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340b17d0 con 0x7fcb340b57b0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30011870 con 0x7fcb340b57b0 2024-01-16T14:55:17.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.998+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb340b1d90 con 0x7fcb340b57b0 2024-01-16T14:55:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:17.999+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34075480 con 0x7fcb340b57b0 2024-01-16T14:55:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.000+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcb30021030 con 0x7fcb340b57b0 2024-01-16T14:55:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.000+0000 7fcb25ffb700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 0x7fcb1806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.000+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb3008b280 con 0x7fcb340b57b0 2024-01-16T14:55:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.001+0000 7fcb39383700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 0x7fcb1806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.001+0000 7fcb39383700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 0x7fcb1806d810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800a8f0 tx=0x7fcb28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.005+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb300560e0 con 0x7fcb340b57b0 2024-01-16T14:55:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.303+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7fcb340afce0 con 0x7fcb340b57b0 2024-01-16T14:55:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.547+0000 7fcb25ffb700 1 -- 172.21.15.26:0/3335406089 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v50) v1 ==== 155+0+0 (secure 0 0 0) 0x7fcb30059700 con 0x7fcb340b57b0 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 msgr2=0x7fcb1806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 0x7fcb1806d810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800a8f0 tx=0x7fcb28008040 comp rx=0 tx=0).stop 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 msgr2=0x7fcb3413f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb340717b0 tx=0x7fcb30004750 comp rx=0 tx=0).stop 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 shutdown_connections 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb341478c0 0x7fcb341466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb3406b510 0x7fcb34155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcb1806b350 0x7fcb1806d810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 --2- 172.21.15.26:0/3335406089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb340b57b0 0x7fcb3413f500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 >> 172.21.15.26:0/3335406089 conn(0x7fcb340d0aa0 msgr2=0x7fcb34071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 shutdown_connections 2024-01-16T14:55:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.551+0000 7fcb3b5e7700 1 -- 172.21.15.26:0/3335406089 wait complete. 2024-01-16T14:55:18.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-16T14:55:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.710+0000 7ff5aaa4a700 1 Processor -- start 2024-01-16T14:55:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.710+0000 7ff5aaa4a700 1 -- start start 2024-01-16T14:55:18.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5aaa4a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 0x7ff5a406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5aaa4a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40cfd60 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5a37fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff5a40d0ae0 0x7ff5a406f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42226/0 (socket says 172.21.15.26:42226) 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5aaa4a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a406f430 con 0x7ff5a406fe00 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5a37fe700 1 -- 172.21.15.26:0/4285001534 learned_addr learned my addr 172.21.15.26:0/4285001534 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.711+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40cf4b0 con 0x7ff5a415e000 2024-01-16T14:55:18.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/4285001534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 0x7ff5a406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a2ffd700 1 -- 172.21.15.26:0/4285001534 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995451745 0 0) 0x7ff5a40cfd60 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a2ffd700 1 -- 172.21.15.26:0/4285001534 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff58c0035b0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a3fff700 1 -- 172.21.15.26:0/4285001534 >> v1:172.21.15.93:6789/0 conn(0x7ff5a406fe00 legacy=0x7ff5a40cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a3fff700 1 -- 172.21.15.26:0/4285001534 >> v1:172.21.15.26:6789/0 conn(0x7ff5a40d0ae0 legacy=0x7ff5a406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a3fff700 1 -- 172.21.15.26:0/4285001534 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40cf1d0 con 0x7ff5a415e000 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/4285001534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 0x7ff5a406eef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff594009870 tx=0x7ff594009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aff12e7346a05e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a2ffd700 1 -- 172.21.15.26:0/4285001534 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594011830 con 0x7ff5a415e000 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a2ffd700 1 -- 172.21.15.26:0/4285001534 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594011990 con 0x7ff5a415e000 2024-01-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.712+0000 7ff5a2ffd700 1 -- 172.21.15.26:0/4285001534 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594011d20 con 0x7ff5a415e000 2024-01-16T14:55:18.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 msgr2=0x7ff5a406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/4285001534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 0x7ff5a406eef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff594009870 tx=0x7ff594009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:18.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 shutdown_connections 2024-01-16T14:55:18.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/4285001534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a415e000 0x7ff5a406eef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 >> 172.21.15.26:0/4285001534 conn(0x7ff5a40b3720 msgr2=0x7ff5a40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:18.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 shutdown_connections 2024-01-16T14:55:18.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.713+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/4285001534 wait complete. 2024-01-16T14:55:18.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.714+0000 7ff5aaa4a700 1 Processor -- start 2024-01-16T14:55:18.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 -- start start 2024-01-16T14:55:18.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a406fe00 0x7ff5a40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:18.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:18.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5a415e000 0x7ff5a40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41f7270 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41f74c0 con 0x7ff5a415e000 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5aaa4a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a41f77a0 con 0x7ff5a406fe00 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58616/0 (socket says 172.21.15.26:58616) 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.715+0000 7ff5a37fe700 1 -- 172.21.15.26:0/1818310141 learned_addr learned my addr 172.21.15.26:0/1818310141 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a406fe00 0x7ff5a40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a8fe7700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5a415e000 0x7ff5a40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 -- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a406fe00 msgr2=0x7ff5a40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a406fe00 0x7ff5a40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 -- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5a415e000 msgr2=0x7ff5a40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5a415e000 0x7ff5a40b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 -- 172.21.15.26:0/1818310141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940094a0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a37fe700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c700 tx=0x7ff59800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598005d70 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980070d0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a41f7a80 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598007460 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.716+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41537e0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.717+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a41f8070 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.718+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff5980075c0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.718+0000 7ff5a0ff9700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 0x7ff59006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:18.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.718+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff598088fa0 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.718+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 0x7ff59006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:18.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.719+0000 7ff5a3fff700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 0x7ff59006d810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5940090b0 tx=0x7ff594008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:18.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.723+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5980bbe90 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.932+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff5a4075480 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.935+0000 7ff5a0ff9700 1 -- 172.21.15.26:0/1818310141 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff598017020 con 0x7ff5a40d0ae0 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.937+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 msgr2=0x7ff59006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.937+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 0x7ff59006d810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5940090b0 tx=0x7ff594008030 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.937+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 msgr2=0x7ff5a41532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.937+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c700 tx=0x7ff59800ca10 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 shutdown_connections 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5a415e000 0x7ff5a40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5a406fe00 0x7ff5a40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff59006b350 0x7ff59006d810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 --2- 172.21.15.26:0/1818310141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5a40d0ae0 0x7ff5a41532a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 >> 172.21.15.26:0/1818310141 conn(0x7ff5a40b3720 msgr2=0x7ff5a40d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 shutdown_connections 2024-01-16T14:55:18.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:18.938+0000 7ff5aaa4a700 1 -- 172.21.15.26:0/1818310141 wait complete. 2024-01-16T14:55:18.935 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-16T14:55:18.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T14:55:18.976 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T14:55:19.027 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-16T14:55:19.027 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:19.027 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:19.048 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:19.048 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:19.110 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:19.110 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:19.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.375+0000 7f1589f86700 1 Processor -- start 2024-01-16T14:55:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.375+0000 7f1589f86700 1 -- start start 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1589f86700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f158415e000 0x7f158406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1589f86700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1589f86700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15840cfd60 con 0x7f15840d0f60 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1589f86700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15840cf4b0 con 0x7f158415e000 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1589f86700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158406f430 con 0x7f15840d0ae0 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f158415e000 0x7f158406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f158415e000 0x7f158406eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58626/0 (socket says 172.21.15.26:58626) 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 -- 172.21.15.26:0/2648432795 learned_addr learned my addr 172.21.15.26:0/2648432795 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1582ffd700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 -- 172.21.15.26:0/2648432795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 msgr2=0x7f158406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 -- 172.21.15.26:0/2648432795 >> v1:172.21.15.93:6789/0 conn(0x7f15840d0f60 legacy=0x7f15840cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f15837fe700 1 -- 172.21.15.26:0/2648432795 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840cf1d0 con 0x7f158415e000 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.376+0000 7f1582ffd700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f15837fe700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f158406eef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15740098f0 tx=0x7f1574009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b9768d514808db server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f15827fc700 1 -- 172.21.15.26:0/2648432795 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574004230 con 0x7f158415e000 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f15827fc700 1 -- 172.21.15.26:0/2648432795 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574004390 con 0x7f158415e000 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f15827fc700 1 -- 172.21.15.26:0/2648432795 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740036a0 con 0x7f158415e000 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 -- 172.21.15.26:0/2648432795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 msgr2=0x7f158406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f158406eef0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15740098f0 tx=0x7f1574009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 -- 172.21.15.26:0/2648432795 shutdown_connections 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f158406eef0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 --2- 172.21.15.26:0/2648432795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.377+0000 7f1589f86700 1 -- 172.21.15.26:0/2648432795 >> 172.21.15.26:0/2648432795 conn(0x7f15840b3720 msgr2=0x7f15840b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.378+0000 7f1589f86700 1 -- 172.21.15.26:0/2648432795 shutdown_connections 2024-01-16T14:55:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.378+0000 7f1589f86700 1 -- 172.21.15.26:0/2648432795 wait complete. 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.379+0000 7f1589f86700 1 Processor -- start 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.379+0000 7f1589f86700 1 -- start start 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 0x7f15840c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b54e0 con 0x7f158415e000 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b57c0 con 0x7f15840d0f60 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1589f86700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15840b5aa0 con 0x7f15840d0ae0 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58640/0 (socket says 172.21.15.26:58640) 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 -- 172.21.15.26:0/2385964758 learned_addr learned my addr 172.21.15.26:0/2385964758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f15837fe700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1582ffd700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 0x7f15840c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 -- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 msgr2=0x7f158414c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158414c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 -- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 msgr2=0x7f15840c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 0x7f15840c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 -- 172.21.15.26:0/2385964758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574009550 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1582ffd700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 0x7f15840c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.380+0000 7f1583fff700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f157c00b970 tx=0x7f157c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.381+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00c930 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.381+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00cad0 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.381+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15840c7290 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.381+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00ce80 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.381+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15840b5d80 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.382+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1584075480 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.382+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f157c014440 con 0x7f158415e000 2024-01-16T14:55:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.383+0000 7f156bfff700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 0x7f157006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.383+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f157c089770 con 0x7f158415e000 2024-01-16T14:55:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.383+0000 7f15837fe700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 0x7f157006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.383+0000 7f15837fe700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 0x7f157006d810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1574009840 tx=0x7f1574008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.388+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f157c0545f0 con 0x7f158415e000 2024-01-16T14:55:19.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.601+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f158415d2b0 con 0x7f158415e000 2024-01-16T14:55:19.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.602+0000 7f156bfff700 1 -- 172.21.15.26:0/2385964758 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f157c057c10 con 0x7f158415e000 2024-01-16T14:55:19.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:19.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 msgr2=0x7f157006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 0x7f157006d810 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1574009840 tx=0x7f1574008030 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 msgr2=0x7f15841605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f157c00b970 tx=0x7f157c00bc80 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 shutdown_connections 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f157006b350 0x7f157006d810 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f158415e000 0x7f15841605b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15840d0ae0 0x7f158414c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 --2- 172.21.15.26:0/2385964758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15840d0f60 0x7f15840c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.604+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 >> 172.21.15.26:0/2385964758 conn(0x7f15840b3720 msgr2=0x7f1584074070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.605+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 shutdown_connections 2024-01-16T14:55:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.605+0000 7f1589f86700 1 -- 172.21.15.26:0/2385964758 wait complete. 2024-01-16T14:55:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:19.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.758+0000 7ff2aec28700 1 Processor -- start 2024-01-16T14:55:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.758+0000 7ff2aec28700 1 -- start start 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2aec28700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 0x7ff2a80b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2aec28700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a8161a90 0x7ff2a806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2aec28700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a806dbe0 con 0x7ff2a806b510 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2aec28700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a806b9a0 con 0x7ff2a8161a90 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2aec28700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a8147ca0 con 0x7ff2a81478c0 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2a7fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a8161a90 0x7ff2a806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2a7fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a8161a90 0x7ff2a806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58654/0 (socket says 172.21.15.26:58654) 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.759+0000 7ff2a7fff700 1 -- 172.21.15.26:0/2968586542 learned_addr learned my addr 172.21.15.26:0/2968586542 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/2968586542 >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 0x7ff2a80b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a7fff700 1 -- 172.21.15.26:0/2968586542 >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 msgr2=0x7ff2a80b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a7fff700 1 --2- 172.21.15.26:0/2968586542 >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 0x7ff2a80b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a7fff700 1 -- 172.21.15.26:0/2968586542 >> v1:172.21.15.93:6789/0 conn(0x7ff2a806b510 legacy=0x7ff2a8071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a7fff700 1 -- 172.21.15.26:0/2968586542 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a81622e0 con 0x7ff2a8161a90 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/2968586542 >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 0x7ff2a80b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a7fff700 1 --2- 172.21.15.26:0/2968586542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a806afd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8072fa0 tx=0x7ff29c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1386a04789cac4 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a77fe700 1 -- 172.21.15.26:0/2968586542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c004230 con 0x7ff2a8161a90 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a77fe700 1 -- 172.21.15.26:0/2968586542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c004390 con 0x7ff2a8161a90 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.760+0000 7ff2a77fe700 1 -- 172.21.15.26:0/2968586542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0036a0 con 0x7ff2a8161a90 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 -- 172.21.15.26:0/2968586542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 msgr2=0x7ff2a806afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 --2- 172.21.15.26:0/2968586542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a806afd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8072fa0 tx=0x7ff29c009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 -- 172.21.15.26:0/2968586542 shutdown_connections 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 --2- 172.21.15.26:0/2968586542 >> v2:172.21.15.93:3300/0 conn(0x7ff2a81478c0 0x7ff2a80b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 --2- 172.21.15.26:0/2968586542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a806afd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 -- 172.21.15.26:0/2968586542 >> 172.21.15.26:0/2968586542 conn(0x7ff2a80d0aa0 msgr2=0x7ff2a80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:19.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 -- 172.21.15.26:0/2968586542 shutdown_connections 2024-01-16T14:55:19.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.761+0000 7ff2aec28700 1 -- 172.21.15.26:0/2968586542 wait complete. 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.762+0000 7ff2aec28700 1 Processor -- start 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.762+0000 7ff2aec28700 1 -- start start 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 0x7ff2a81607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 0x7ff2a813c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8160d00 con 0x7ff2a8161a90 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a813d2c0 con 0x7ff2a806b510 2024-01-16T14:55:19.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2aec28700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a813d5a0 con 0x7ff2a81478c0 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2ad1c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2ad1c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58666/0 (socket says 172.21.15.26:58666) 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.763+0000 7ff2ad1c5700 1 -- 172.21.15.26:0/3010735026 learned_addr learned my addr 172.21.15.26:0/3010735026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 0x7ff2a81607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2a7fff700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 0x7ff2a813c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 -- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 msgr2=0x7ff2a813c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 0x7ff2a813c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 -- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 msgr2=0x7ff2a81607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 0x7ff2a81607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 -- 172.21.15.26:0/3010735026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009550 con 0x7ff2a8161a90 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 0x7ff2a81607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2a7fff700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 0x7ff2a813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2ad1c5700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000cc10 tx=0x7ff2a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000abb0 con 0x7ff2a8161a90 2024-01-16T14:55:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000ad10 con 0x7ff2a8161a90 2024-01-16T14:55:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a813d880 con 0x7ff2a8161a90 2024-01-16T14:55:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0012660 con 0x7ff2a8161a90 2024-01-16T14:55:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.764+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a813de40 con 0x7ff2a8161a90 2024-01-16T14:55:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.765+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a8075480 con 0x7ff2a8161a90 2024-01-16T14:55:19.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.766+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff2a00127c0 con 0x7ff2a8161a90 2024-01-16T14:55:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.766+0000 7ff2a57fa700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 0x7ff29406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.766+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2a008a390 con 0x7ff2a8161a90 2024-01-16T14:55:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.766+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 0x7ff29406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:19.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.767+0000 7ff2ac9c4700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 0x7ff29406d810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff298006fd0 tx=0x7ff298008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:19.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.771+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2a0055170 con 0x7ff2a8161a90 2024-01-16T14:55:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.982+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2a806bcb0 con 0x7ff2a8161a90 2024-01-16T14:55:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.983+0000 7ff2a57fa700 1 -- 172.21.15.26:0/3010735026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2a0016020 con 0x7ff2a8161a90 2024-01-16T14:55:19.967 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 msgr2=0x7ff29406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 0x7ff29406d810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff298006fd0 tx=0x7ff298008040 comp rx=0 tx=0).stop 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 msgr2=0x7ff2a813e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000cc10 tx=0x7ff2a000a060 comp rx=0 tx=0).stop 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 shutdown_connections 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff29406b350 0x7ff29406d810 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a8161a90 0x7ff2a813e360 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2a806b510 0x7ff2a81607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 --2- 172.21.15.26:0/3010735026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2a81478c0 0x7ff2a813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 >> 172.21.15.26:0/3010735026 conn(0x7ff2a80d0aa0 msgr2=0x7ff2a806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 shutdown_connections 2024-01-16T14:55:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:19.985+0000 7ff2aec28700 1 -- 172.21.15.26:0/3010735026 wait complete. 2024-01-16T14:55:19.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.137+0000 7fb3b73da700 1 Processor -- start 2024-01-16T14:55:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.138+0000 7fb3b73da700 1 -- start start 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.138+0000 7fb3b73da700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb3b015bde0 0x7fb3b015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b73da700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3b00ce5c0 0x7fb3b00cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b73da700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00d0a90 con 0x7fb3b00cf2b0 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b73da700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00cef70 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b73da700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b006ee10 con 0x7fb3b015bde0 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3b00ce5c0 0x7fb3b00cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b4975700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb3b00cf2b0 0x7fb3b00d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42234/0 (socket says 172.21.15.26:42234) 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b4975700 1 -- 172.21.15.26:0/17558093 learned_addr learned my addr 172.21.15.26:0/17558093 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5176700 1 --2- 172.21.15.26:0/17558093 >> v2:172.21.15.93:3300/0 conn(0x7fb3b015bde0 0x7fb3b015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 -- 172.21.15.26:0/17558093 >> v2:172.21.15.93:3300/0 conn(0x7fb3b015bde0 msgr2=0x7fb3b015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 --2- 172.21.15.26:0/17558093 >> v2:172.21.15.93:3300/0 conn(0x7fb3b015bde0 0x7fb3b015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 -- 172.21.15.26:0/17558093 >> v1:172.21.15.26:6789/0 conn(0x7fb3b00cf2b0 legacy=0x7fb3b00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 -- 172.21.15.26:0/17558093 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b006f0f0 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3a7fff700 1 -- 172.21.15.26:0/17558093 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052331479 0 0) 0x7fb3b00d0a90 con 0x7fb3b00cf2b0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3b5977700 1 --2- 172.21.15.26:0/17558093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00cea30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0072370 tx=0x7fb3a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffc2287448db43 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.139+0000 7fb3a7fff700 1 -- 172.21.15.26:0/17558093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800e070 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3a7fff700 1 -- 172.21.15.26:0/17558093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a800bbd0 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3a7fff700 1 -- 172.21.15.26:0/17558093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8012430 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 -- 172.21.15.26:0/17558093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 msgr2=0x7fb3b00cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 --2- 172.21.15.26:0/17558093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00cea30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0072370 tx=0x7fb3a800b330 comp rx=0 tx=0).stop 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 -- 172.21.15.26:0/17558093 shutdown_connections 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 --2- 172.21.15.26:0/17558093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00cea30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 --2- 172.21.15.26:0/17558093 >> v2:172.21.15.93:3300/0 conn(0x7fb3b015bde0 0x7fb3b015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 -- 172.21.15.26:0/17558093 >> 172.21.15.26:0/17558093 conn(0x7fb3b00b3720 msgr2=0x7fb3b00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:20.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 -- 172.21.15.26:0/17558093 shutdown_connections 2024-01-16T14:55:20.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.140+0000 7fb3b73da700 1 -- 172.21.15.26:0/17558093 wait complete. 2024-01-16T14:55:20.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 Processor -- start 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 -- start start 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 0x7fb3b01f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b015bde0 0x7fb3b01f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cc020 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01f47f0 con 0x7fb3b015bde0 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b73da700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01f4510 con 0x7fb3b00cf2b0 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.142+0000 7fb3b5176700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58690/0 (socket says 172.21.15.26:58690) 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 -- 172.21.15.26:0/1541780435 learned_addr learned my addr 172.21.15.26:0/1541780435 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5977700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b015bde0 0x7fb3b01f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b4975700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 0x7fb3b01f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 -- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 msgr2=0x7fb3b01f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 0x7fb3b01f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 -- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b015bde0 msgr2=0x7fb3b01f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b015bde0 0x7fb3b01f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 -- 172.21.15.26:0/1541780435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8009a70 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b4975700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 0x7fb3b01f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b5176700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00d800 tx=0x7fb3ac00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0125f0 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac012750 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b0147d40 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.143+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac012b00 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.144+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b0148300 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.145+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0148490 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.145+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb3ac012c60 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.145+0000 7fb3a5ffb700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 0x7fb39406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.145+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3ac089f40 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.145+0000 7fb3b4975700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 0x7fb39406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.146+0000 7fb3b4975700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 0x7fb39406d810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01459e0 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.150+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3ac054db0 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.361+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3b0075480 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.362+0000 7fb3a5ffb700 1 -- 172.21.15.26:0/1541780435 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3ac0583d0 con 0x7fb3b00ce5c0 2024-01-16T14:55:20.346 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 msgr2=0x7fb39406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 0x7fb39406d810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01459e0 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2024-01-16T14:55:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 msgr2=0x7fb3b00b6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00d800 tx=0x7fb3ac00dbc0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 shutdown_connections 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3b015bde0 0x7fb3b01f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb39406b350 0x7fb39406d810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3b00ce5c0 0x7fb3b00b6a90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 --2- 172.21.15.26:0/1541780435 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb3b00cf2b0 0x7fb3b01f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 >> 172.21.15.26:0/1541780435 conn(0x7fb3b00b3720 msgr2=0x7fb3b015e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 shutdown_connections 2024-01-16T14:55:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.364+0000 7fb3b73da700 1 -- 172.21.15.26:0/1541780435 wait complete. 2024-01-16T14:55:20.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.518+0000 7f9c98a51700 1 Processor -- start 2024-01-16T14:55:20.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.518+0000 7f9c98a51700 1 -- start start 2024-01-16T14:55:20.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9c94072940 0x7f9c9406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c9406c350 0x7f9c94066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9406d030 con 0x7f9c9406c350 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c940b29a0 con 0x7f9c94072940 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c98a51700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9406efb0 con 0x7f9c94072d20 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c91d9b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c9406c350 0x7f9c94066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c91d9b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c9406c350 0x7f9c94066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58704/0 (socket says 172.21.15.26:58704) 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.519+0000 7f9c91d9b700 1 -- 172.21.15.26:0/3972270770 learned_addr learned my addr 172.21.15.26:0/3972270770 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c9259c700 1 --2- 172.21.15.26:0/3972270770 >> v2:172.21.15.93:3300/0 conn(0x7f9c94072940 0x7f9c9406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c92d9d700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 -- 172.21.15.26:0/3972270770 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 msgr2=0x7f9c9415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 -- 172.21.15.26:0/3972270770 >> v2:172.21.15.93:3300/0 conn(0x7f9c94072940 msgr2=0x7f9c9406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/3972270770 >> v2:172.21.15.93:3300/0 conn(0x7f9c94072940 0x7f9c9406d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 -- 172.21.15.26:0/3972270770 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9406c850 con 0x7f9c9406c350 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c94066ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c94071820 tx=0x7f9c88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51e53235f530054 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c9159a700 1 -- 172.21.15.26:0/3972270770 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800b870 con 0x7f9c9406c350 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c9159a700 1 -- 172.21.15.26:0/3972270770 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8800b9d0 con 0x7f9c9406c350 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.520+0000 7f9c9159a700 1 -- 172.21.15.26:0/3972270770 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800bd60 con 0x7f9c9406c350 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 -- 172.21.15.26:0/3972270770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 msgr2=0x7f9c94066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c94066ce0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c94071820 tx=0x7f9c88009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 -- 172.21.15.26:0/3972270770 shutdown_connections 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 --2- 172.21.15.26:0/3972270770 >> v2:172.21.15.93:3300/0 conn(0x7f9c94072940 0x7f9c9406d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 --2- 172.21.15.26:0/3972270770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c94066ce0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 -- 172.21.15.26:0/3972270770 >> 172.21.15.26:0/3972270770 conn(0x7f9c940b1e80 msgr2=0x7f9c940b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 -- 172.21.15.26:0/3972270770 shutdown_connections 2024-01-16T14:55:20.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.521+0000 7f9c98a51700 1 -- 172.21.15.26:0/3972270770 wait complete. 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.522+0000 7f9c98a51700 1 Processor -- start 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.522+0000 7f9c98a51700 1 -- start start 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c94072940 0x7f9c940b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940b0820 con 0x7f9c9406c350 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940cc190 con 0x7f9c94072940 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c98a51700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940cb860 con 0x7f9c94072d20 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c9259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c9259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58716/0 (socket says 172.21.15.26:58716) 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c9259c700 1 -- 172.21.15.26:0/1551611647 learned_addr learned my addr 172.21.15.26:0/1551611647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c94072940 0x7f9c940b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.523+0000 7f9c92d9d700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 -- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 msgr2=0x7f9c9414fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9414fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 -- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c94072940 msgr2=0x7f9c940b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c94072940 0x7f9c940b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 -- 172.21.15.26:0/1551611647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88009550 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c92d9d700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c9259c700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400e810 tx=0x7f9c8400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84013610 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84013790 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84013b40 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c940cbb40 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.524+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9409a8e0 con 0x7f9c9406c350 2024-01-16T14:55:20.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.525+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c94075480 con 0x7f9c9406c350 2024-01-16T14:55:20.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.526+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9c84013ca0 con 0x7f9c9406c350 2024-01-16T14:55:20.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.526+0000 7f9c82ffd700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 0x7f9c7406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.526+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c84089680 con 0x7f9c9406c350 2024-01-16T14:55:20.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.527+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 0x7f9c7406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.527+0000 7f9c91d9b700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 0x7f9c7406d810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c94071820 tx=0x7f9c88008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.531+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c84054580 con 0x7f9c9406c350 2024-01-16T14:55:20.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.740+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c9415d9e0 con 0x7f9c9406c350 2024-01-16T14:55:20.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.741+0000 7f9c82ffd700 1 -- 172.21.15.26:0/1551611647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c84009440 con 0x7f9c9406c350 2024-01-16T14:55:20.725 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 msgr2=0x7f9c7406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 0x7f9c7406d810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c94071820 tx=0x7f9c88008030 comp rx=0 tx=0).stop 2024-01-16T14:55:20.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 msgr2=0x7f9c9409a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c8400e810 tx=0x7f9c8400eb20 comp rx=0 tx=0).stop 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 shutdown_connections 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c94072d20 0x7f9c9414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9c7406b350 0x7f9c7406d810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9406c350 0x7f9c9409a270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 --2- 172.21.15.26:0/1551611647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c94072940 0x7f9c940b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 >> 172.21.15.26:0/1551611647 conn(0x7f9c940b1e80 msgr2=0x7f9c940c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 shutdown_connections 2024-01-16T14:55:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.743+0000 7f9c98a51700 1 -- 172.21.15.26:0/1551611647 wait complete. 2024-01-16T14:55:20.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T14:55:20.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.894+0000 7f2b0ded5700 1 Processor -- start 2024-01-16T14:55:20.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.894+0000 7f2b0ded5700 1 -- start start 2024-01-16T14:55:20.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b0815e000 0x7f2b0806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b080d0ae0 0x7f2b0806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2b080d0f60 0x7f2b0806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080cf4b0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080cfd60 con 0x7f2b080d0f60 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b0ded5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0806f430 con 0x7f2b0815e000 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b06ffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b080d0ae0 0x7f2b0806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b06ffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b080d0ae0 0x7f2b0806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58730/0 (socket says 172.21.15.26:58730) 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b06ffd700 1 -- 172.21.15.26:0/1366782064 learned_addr learned my addr 172.21.15.26:0/1366782064 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b07fff700 1 --2- 172.21.15.26:0/1366782064 >> v2:172.21.15.93:3300/0 conn(0x7f2b080d0f60 0x7f2b0806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.895+0000 7f2b077fe700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b0815e000 0x7f2b0806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 -- 172.21.15.26:0/1366782064 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b0815e000 msgr2=0x7f2b0806eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b0815e000 0x7f2b0806eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 -- 172.21.15.26:0/1366782064 >> v2:172.21.15.93:3300/0 conn(0x7f2b080d0f60 msgr2=0x7f2b0806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/1366782064 >> v2:172.21.15.93:3300/0 conn(0x7f2b080d0f60 0x7f2b0806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 -- 172.21.15.26:0/1366782064 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080cf1d0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b0806f6f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b08073110 tx=0x7f2afc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec4101cb8ece730 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b067fc700 1 -- 172.21.15.26:0/1366782064 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00b870 con 0x7f2b080d0ae0 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b067fc700 1 -- 172.21.15.26:0/1366782064 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc00b9d0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b067fc700 1 -- 172.21.15.26:0/1366782064 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00bd60 con 0x7f2b080d0ae0 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b0ded5700 1 -- 172.21.15.26:0/1366782064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 msgr2=0x7f2b0806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.896+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b0806f6f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b08073110 tx=0x7f2afc009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 -- 172.21.15.26:0/1366782064 shutdown_connections 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/1366782064 >> v2:172.21.15.93:3300/0 conn(0x7f2b080d0f60 0x7f2b0806fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b0815e000 0x7f2b0806eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/1366782064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b0806f6f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 -- 172.21.15.26:0/1366782064 >> 172.21.15.26:0/1366782064 conn(0x7f2b080b3720 msgr2=0x7f2b080b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 -- 172.21.15.26:0/1366782064 shutdown_connections 2024-01-16T14:55:20.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.897+0000 7f2b0ded5700 1 -- 172.21.15.26:0/1366782064 wait complete. 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.898+0000 7f2b0ded5700 1 Processor -- start 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.898+0000 7f2b0ded5700 1 -- start start 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b080d0f60 0x7f2b08147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b0815e000 0x7f2b080b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080af310 con 0x7f2b080d0ae0 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080b7920 con 0x7f2b0815e000 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b0ded5700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080af5d0 con 0x7f2b080d0f60 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58744/0 (socket says 172.21.15.26:58744) 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 -- 172.21.15.26:0/2230515307 learned_addr learned my addr 172.21.15.26:0/2230515307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b07fff700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b0815e000 0x7f2b080b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b080d0f60 0x7f2b08147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 -- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b080d0f60 msgr2=0x7f2b08147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b080d0f60 0x7f2b08147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 -- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b0815e000 msgr2=0x7f2b080b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b0815e000 0x7f2b080b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.899+0000 7f2b077fe700 1 -- 172.21.15.26:0/2230515307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc009550 con 0x7f2b080d0ae0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2b077fe700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2af800d800 tx=0x7f2af800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80125f0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8012750 con 0x7f2b080d0ae0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b080af8b0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8012b00 con 0x7f2b080d0ae0 2024-01-16T14:55:20.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.900+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b080afe70 con 0x7f2b080d0ae0 2024-01-16T14:55:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.901+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b080b01e0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.901+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2af8012c60 con 0x7f2b080d0ae0 2024-01-16T14:55:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.902+0000 7f2aeffff700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 0x7f2af406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.902+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2af8089bc0 con 0x7f2b080d0ae0 2024-01-16T14:55:20.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.902+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 0x7f2af406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:20.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.903+0000 7f2b06ffd700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 0x7f2af406d810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009840 tx=0x7f2afc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:20.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:20.907+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2af8054a30 con 0x7f2b080d0ae0 2024-01-16T14:55:21.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.135+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b08075480 con 0x7f2b080d0ae0 2024-01-16T14:55:21.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.135+0000 7f2aeffff700 1 -- 172.21.15.26:0/2230515307 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2af8058050 con 0x7f2b080d0ae0 2024-01-16T14:55:21.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 msgr2=0x7f2af406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 0x7f2af406d810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009840 tx=0x7f2afc008030 comp rx=0 tx=0).stop 2024-01-16T14:55:21.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 msgr2=0x7f2b080b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2af800d800 tx=0x7f2af800db10 comp rx=0 tx=0).stop 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 shutdown_connections 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b0815e000 0x7f2b080b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2af406b350 0x7f2af406d810 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b080d0ae0 0x7f2b080b73e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 --2- 172.21.15.26:0/2230515307 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b080d0f60 0x7f2b08147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.137+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 >> 172.21.15.26:0/2230515307 conn(0x7f2b080b3720 msgr2=0x7f2b0815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.138+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 shutdown_connections 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.138+0000 7f2b0ded5700 1 -- 172.21.15.26:0/2230515307 wait complete. 2024-01-16T14:55:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-16T14:55:21.135 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-16T14:55:21.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:21.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-01-16T14:55:21.185 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:55:21.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:21.186 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-16T14:55:21.217 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:21.217 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:21.239 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:21.239 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:21.299 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:21.300 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:21.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.522+0000 7f3928228700 1 Processor -- start 2024-01-16T14:55:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.522+0000 7f3928228700 1 -- start start 2024-01-16T14:55:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3928228700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f392006afd0 0x7f39200d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3928228700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39200ada10 0x7f39200adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3928228700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3920070490 con 0x7f39200affb0 2024-01-16T14:55:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3928228700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200b0410 con 0x7f392006afd0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3928228700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200af9f0 con 0x7f39200ada10 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3925fc4700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f392006afd0 0x7f39200d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f39257c3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f39200affb0 0x7f392006b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42242/0 (socket says 172.21.15.26:42242) 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f39257c3700 1 -- 172.21.15.26:0/1068688153 learned_addr learned my addr 172.21.15.26:0/1068688153 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f39267c5700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39200ada10 0x7f39200adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3925fc4700 1 -- 172.21.15.26:0/1068688153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39200ada10 msgr2=0x7f39200adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3925fc4700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39200ada10 0x7f39200adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3925fc4700 1 -- 172.21.15.26:0/1068688153 >> v1:172.21.15.26:6789/0 conn(0x7f39200affb0 legacy=0x7f392006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3925fc4700 1 -- 172.21.15.26:0/1068688153 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200afcd0 con 0x7f392006afd0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.523+0000 7f3924fc2700 1 -- 172.21.15.26:0/1068688153 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857453084 0 0) 0x7f3920070490 con 0x7f39200affb0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3925fc4700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392006afd0 0x7f39200d16e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f391c009840 tx=0x7f391c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcba2c81d2aba28 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3924fc2700 1 -- 172.21.15.26:0/1068688153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c011830 con 0x7f392006afd0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3924fc2700 1 -- 172.21.15.26:0/1068688153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c011990 con 0x7f392006afd0 2024-01-16T14:55:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3924fc2700 1 -- 172.21.15.26:0/1068688153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c011d20 con 0x7f392006afd0 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 -- 172.21.15.26:0/1068688153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392006afd0 msgr2=0x7f39200d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392006afd0 0x7f39200d16e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f391c009840 tx=0x7f391c009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 -- 172.21.15.26:0/1068688153 shutdown_connections 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39200ada10 0x7f39200adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 --2- 172.21.15.26:0/1068688153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392006afd0 0x7f39200d16e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.524+0000 7f3928228700 1 -- 172.21.15.26:0/1068688153 >> 172.21.15.26:0/1068688153 conn(0x7f39200d0bc0 msgr2=0x7f39200d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.525+0000 7f3928228700 1 -- 172.21.15.26:0/1068688153 shutdown_connections 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.525+0000 7f3928228700 1 -- 172.21.15.26:0/1068688153 wait complete. 2024-01-16T14:55:21.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.526+0000 7f3928228700 1 Processor -- start 2024-01-16T14:55:21.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.526+0000 7f3928228700 1 -- start start 2024-01-16T14:55:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392006afd0 0x7f392014c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39200ada10 0x7f3920148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3920149850 con 0x7f39200affb0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392014c750 con 0x7f39200ada10 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.527+0000 7f3928228700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200d0190 con 0x7f392006afd0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58758/0 (socket says 172.21.15.26:58758) 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 -- 172.21.15.26:0/850739368 learned_addr learned my addr 172.21.15.26:0/850739368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f3925fc4700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392006afd0 0x7f392014c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39257c3700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39200ada10 0x7f3920148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 -- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392006afd0 msgr2=0x7f392014c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392006afd0 0x7f392014c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 -- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39200ada10 msgr2=0x7f3920148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39200ada10 0x7f3920148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 -- 172.21.15.26:0/850739368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0094a0 con 0x7f39200affb0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39267c5700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3918007e40 tx=0x7f391800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800ed30 con 0x7f39200affb0 2024-01-16T14:55:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918014420 con 0x7f39200affb0 2024-01-16T14:55:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392015f470 con 0x7f39200affb0 2024-01-16T14:55:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180147d0 con 0x7f39200affb0 2024-01-16T14:55:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.528+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3920149da0 con 0x7f39200affb0 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.529+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920075480 con 0x7f39200affb0 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.530+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3918014930 con 0x7f39200affb0 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.530+0000 7f39127fc700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 0x7f390406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.530+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f391808a2e0 con 0x7f39200affb0 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.531+0000 7f3925fc4700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 0x7f390406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.531+0000 7f3925fc4700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 0x7f390406d810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391c0090b0 tx=0x7f391c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.536+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39180550b0 con 0x7f39200affb0 2024-01-16T14:55:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.748+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39200b0aa0 con 0x7f39200affb0 2024-01-16T14:55:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.749+0000 7f39127fc700 1 -- 172.21.15.26:0/850739368 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f39180586d0 con 0x7f39200affb0 2024-01-16T14:55:21.733 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:21.734 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 msgr2=0x7f390406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 0x7f390406d810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391c0090b0 tx=0x7f391c01c040 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 msgr2=0x7f39200ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3918007e40 tx=0x7f391800e3c0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 shutdown_connections 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f390406b350 0x7f390406d810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39200affb0 0x7f39200ae440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392006afd0 0x7f392014c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 --2- 172.21.15.26:0/850739368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39200ada10 0x7f3920148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 >> 172.21.15.26:0/850739368 conn(0x7f39200d0bc0 msgr2=0x7f3920071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 shutdown_connections 2024-01-16T14:55:21.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.752+0000 7f3928228700 1 -- 172.21.15.26:0/850739368 wait complete. 2024-01-16T14:55:21.749 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:21.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.906+0000 7f08b636c700 1 Processor -- start 2024-01-16T14:55:21.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.906+0000 7f08b636c700 1 -- start start 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b636c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08b0071bb0 0x7f08b0072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b636c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f08b00b3c60 0x7f08b00b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b636c700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b015e990 con 0x7f08b0158730 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b636c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b0072570 con 0x7f08b0071bb0 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b636c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b015ec70 con 0x7f08b00b3c60 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08b0071bb0 0x7f08b0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08b0071bb0 0x7f08b0072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58760/0 (socket says 172.21.15.26:58760) 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 -- 172.21.15.26:0/111299350 learned_addr learned my addr 172.21.15.26:0/111299350 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08b4909700 1 --2- 172.21.15.26:0/111299350 >> v2:172.21.15.93:3300/0 conn(0x7f08b00b3c60 0x7f08b00b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 -- 172.21.15.26:0/111299350 >> v2:172.21.15.93:3300/0 conn(0x7f08b00b3c60 msgr2=0x7f08b00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 --2- 172.21.15.26:0/111299350 >> v2:172.21.15.93:3300/0 conn(0x7f08b00b3c60 0x7f08b00b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 -- 172.21.15.26:0/111299350 >> v1:172.21.15.93:6789/0 conn(0x7f08b0158730 legacy=0x7f08b006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 -- 172.21.15.26:0/111299350 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b015ef50 con 0x7f08b0071bb0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08af7fe700 1 --2- 172.21.15.26:0/111299350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b0072030 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08b015b200 tx=0x7f08a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172d0fdb42244160 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08aeffd700 1 -- 172.21.15.26:0/111299350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b870 con 0x7f08b0071bb0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08aeffd700 1 -- 172.21.15.26:0/111299350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400b9d0 con 0x7f08b0071bb0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.907+0000 7f08aeffd700 1 -- 172.21.15.26:0/111299350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400bd60 con 0x7f08b0071bb0 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 -- 172.21.15.26:0/111299350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 msgr2=0x7f08b0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 --2- 172.21.15.26:0/111299350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b0072030 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f08b015b200 tx=0x7f08a4009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 -- 172.21.15.26:0/111299350 shutdown_connections 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 --2- 172.21.15.26:0/111299350 >> v2:172.21.15.93:3300/0 conn(0x7f08b00b3c60 0x7f08b00b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 --2- 172.21.15.26:0/111299350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b0072030 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 -- 172.21.15.26:0/111299350 >> 172.21.15.26:0/111299350 conn(0x7f08b00b27b0 msgr2=0x7f08b00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 -- 172.21.15.26:0/111299350 shutdown_connections 2024-01-16T14:55:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.908+0000 7f08b636c700 1 -- 172.21.15.26:0/111299350 wait complete. 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.909+0000 7f08b636c700 1 Processor -- start 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 -- start start 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08b00b3c60 0x7f08b015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08b0158730 0x7f08b015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b013fce0 con 0x7f08b0071bb0 2024-01-16T14:55:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.910+0000 7f08b636c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b01598e0 con 0x7f08b00b3c60 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08b636c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b00b1bd0 con 0x7f08b0158730 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58762/0 (socket says 172.21.15.26:58762) 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 -- 172.21.15.26:0/846062577 learned_addr learned my addr 172.21.15.26:0/846062577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08b4909700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08b0158730 0x7f08b015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08af7fe700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08b00b3c60 0x7f08b015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 -- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08b0158730 msgr2=0x7f08b015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08b0158730 0x7f08b015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 -- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08b00b3c60 msgr2=0x7f08b015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08b00b3c60 0x7f08b015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 -- 172.21.15.26:0/846062577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009550 con 0x7f08b0071bb0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08affff700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08a000c450 tx=0x7f08a000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00052d0 con 0x7f08b0071bb0 2024-01-16T14:55:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0005430 con 0x7f08b0071bb0 2024-01-16T14:55:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b00b1eb0 con 0x7f08b0071bb0 2024-01-16T14:55:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00038c0 con 0x7f08b0071bb0 2024-01-16T14:55:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.911+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b0159570 con 0x7f08b0071bb0 2024-01-16T14:55:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.912+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b0075480 con 0x7f08b0071bb0 2024-01-16T14:55:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.913+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f08a0003a20 con 0x7f08b0071bb0 2024-01-16T14:55:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.913+0000 7f08acff9700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 0x7f089c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.913+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a0088f80 con 0x7f08b0071bb0 2024-01-16T14:55:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.913+0000 7f08af7fe700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 0x7f089c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.914+0000 7f08af7fe700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 0x7f089c06d810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:21.918+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08a00bbe90 con 0x7f08b0071bb0 2024-01-16T14:55:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.125+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08b00b4de0 con 0x7f08b0071bb0 2024-01-16T14:55:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.126+0000 7f08acff9700 1 -- 172.21.15.26:0/846062577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08a0053d60 con 0x7f08b0071bb0 2024-01-16T14:55:22.109 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 msgr2=0x7f089c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 0x7f089c06d810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4008030 comp rx=0 tx=0).stop 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 msgr2=0x7f08b00cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f08a000c450 tx=0x7f08a000c810 comp rx=0 tx=0).stop 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 shutdown_connections 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08b0158730 0x7f08b015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f089c06b350 0x7f089c06d810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b0071bb0 0x7f08b00cce50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 --2- 172.21.15.26:0/846062577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08b00b3c60 0x7f08b015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 >> 172.21.15.26:0/846062577 conn(0x7f08b00b27b0 msgr2=0x7f08b00b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 shutdown_connections 2024-01-16T14:55:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.128+0000 7f08b636c700 1 -- 172.21.15.26:0/846062577 wait complete. 2024-01-16T14:55:22.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:22.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.281+0000 7ff8c600c700 1 Processor -- start 2024-01-16T14:55:22.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.281+0000 7ff8c600c700 1 -- start start 2024-01-16T14:55:22.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8c600c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c015f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8c600c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c015e930 con 0x7ff8c01589a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8c600c700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c015f8c0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8c600c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0158d80 con 0x7ff8c006dae0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8bf7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8c01589a0 0x7ff8c006f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42256/0 (socket says 172.21.15.26:42256) 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8bf7fe700 1 -- 172.21.15.26:0/853960909 learned_addr learned my addr 172.21.15.26:0/853960909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8beffd700 1 --2- 172.21.15.26:0/853960909 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722249252 0 0) 0x7ff8c015e930 con 0x7ff8c01589a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a80035b0 con 0x7ff8c01589a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148010158 0 0) 0x7ff8c015f8c0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c015e930 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2343591071 0 0) 0x7ff8a80035b0 con 0x7ff8c01589a0 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8beffd700 1 -- 172.21.15.26:0/853960909 >> v1:172.21.15.93:6789/0 conn(0x7ff8c00cd5a0 legacy=0x7ff8c00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8beffd700 1 -- 172.21.15.26:0/853960909 >> v1:172.21.15.26:6789/0 conn(0x7ff8c01589a0 legacy=0x7ff8c006f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.282+0000 7ff8beffd700 1 -- 172.21.15.26:0/853960909 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c015be60 con 0x7ff8c006dae0 2024-01-16T14:55:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8beffd700 1 --2- 172.21.15.26:0/853960909 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c015f380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009870 tx=0x7ff8b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47affc91c759803 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0001670 con 0x7ff8c006dae0 2024-01-16T14:55:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4004030 con 0x7ff8c006dae0 2024-01-16T14:55:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8be7fc700 1 -- 172.21.15.26:0/853960909 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40043a0 con 0x7ff8c006dae0 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 -- 172.21.15.26:0/853960909 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 msgr2=0x7ff8c015f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 --2- 172.21.15.26:0/853960909 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c015f380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009870 tx=0x7ff8b4009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 -- 172.21.15.26:0/853960909 shutdown_connections 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 --2- 172.21.15.26:0/853960909 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c015f380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 -- 172.21.15.26:0/853960909 >> 172.21.15.26:0/853960909 conn(0x7ff8c00d0b30 msgr2=0x7ff8c00d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 -- 172.21.15.26:0/853960909 shutdown_connections 2024-01-16T14:55:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.283+0000 7ff8c600c700 1 -- 172.21.15.26:0/853960909 wait complete. 2024-01-16T14:55:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.285+0000 7ff8c600c700 1 Processor -- start 2024-01-16T14:55:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.285+0000 7ff8c600c700 1 -- start start 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 0x7ff8c0151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01548c0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0154ba0 con 0x7ff8c01589a0 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.286+0000 7ff8c600c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0154e80 con 0x7ff8c006dae0 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58778/0 (socket says 172.21.15.26:58778) 2024-01-16T14:55:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 -- 172.21.15.26:0/495757828 learned_addr learned my addr 172.21.15.26:0/495757828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8bffff700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 0x7ff8c0151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8bf7fe700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 -- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 msgr2=0x7ff8c014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 -- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 msgr2=0x7ff8c0151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 0x7ff8c0151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 -- 172.21.15.26:0/495757828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40094a0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8bffff700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 0x7ff8c0151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8bf7fe700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8beffd700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40090b0 tx=0x7ff8b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4004590 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b401f070 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c0155160 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.287+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4011870 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.288+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c014daa0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.288+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c0075480 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.289+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff8b40119d0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.289+0000 7ff8a7fff700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 0x7ff8ac06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.289+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b4021070 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.290+0000 7ff8bf7fe700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 0x7ff8ac06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.290+0000 7ff8bf7fe700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 0x7ff8ac06d810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005990 tx=0x7ff8b00014d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.294+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8b40564d0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.503+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8c00d16f0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.504+0000 7ff8a7fff700 1 -- 172.21.15.26:0/495757828 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8b4059af0 con 0x7ff8c00cd5a0 2024-01-16T14:55:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 msgr2=0x7ff8ac06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 0x7ff8ac06d810 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005990 tx=0x7ff8b00014d0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 msgr2=0x7ff8c01567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40090b0 tx=0x7ff8b4004750 comp rx=0 tx=0).stop 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 shutdown_connections 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8c01589a0 0x7ff8c0151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8c006dae0 0x7ff8c014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff8ac06b350 0x7ff8ac06d810 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 --2- 172.21.15.26:0/495757828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c00cd5a0 0x7ff8c01567c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 >> 172.21.15.26:0/495757828 conn(0x7ff8c00d0b30 msgr2=0x7ff8c0073f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:22.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 shutdown_connections 2024-01-16T14:55:22.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.506+0000 7ff8c600c700 1 -- 172.21.15.26:0/495757828 wait complete. 2024-01-16T14:55:22.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.660+0000 7f1b7af8d700 1 Processor -- start 2024-01-16T14:55:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.660+0000 7f1b7af8d700 1 -- start start 2024-01-16T14:55:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.661+0000 7f1b7af8d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.661+0000 7f1b7af8d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.661+0000 7f1b7af8d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7415f6b0 con 0x7f1b740b08f0 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.661+0000 7f1b7af8d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b740b0d80 con 0x7f1b7415afc0 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.661+0000 7f1b7af8d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7415eb40 con 0x7f1b7406c5d0 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b78d29700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:45866/0 (socket says 172.21.15.26:45866) 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b7952a700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f1b740b08f0 0x7f1b7406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:34086/0 (socket says 172.21.15.26:34086) 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 -- 172.21.15.26:0/1668501676 learned_addr learned my addr 172.21.15.26:0/1668501676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b78d29700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:34628/0 (socket says 172.21.15.26:34628) 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 -- 172.21.15.26:0/1668501676 >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 msgr2=0x7f1b740730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 --2- 172.21.15.26:0/1668501676 >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 -- 172.21.15.26:0/1668501676 >> v1:172.21.15.93:6789/0 conn(0x7f1b740b08f0 legacy=0x7f1b7406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 -- 172.21.15.26:0/1668501676 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740735f0 con 0x7f1b7406c5d0 2024-01-16T14:55:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b78d29700 1 --2- 172.21.15.26:0/1668501676 >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.662+0000 7f1b73fff700 1 --2- 172.21.15.26:0/1668501676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0098f0 tx=0x7f1b5c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa7bbaca9873b02 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b737fe700 1 -- 172.21.15.26:0/1668501676 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011830 con 0x7f1b7406c5d0 2024-01-16T14:55:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b737fe700 1 -- 172.21.15.26:0/1668501676 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c011990 con 0x7f1b7406c5d0 2024-01-16T14:55:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b737fe700 1 -- 172.21.15.26:0/1668501676 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c011d20 con 0x7f1b7406c5d0 2024-01-16T14:55:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 -- 172.21.15.26:0/1668501676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 msgr2=0x7f1b740b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/1668501676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0098f0 tx=0x7f1b5c009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 -- 172.21.15.26:0/1668501676 shutdown_connections 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/1668501676 >> v2:172.21.15.93:3300/0 conn(0x7f1b7415afc0 0x7f1b740730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/1668501676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b740b03b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 -- 172.21.15.26:0/1668501676 >> 172.21.15.26:0/1668501676 conn(0x7f1b740717f0 msgr2=0x7f1b740b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 -- 172.21.15.26:0/1668501676 shutdown_connections 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.663+0000 7f1b7af8d700 1 -- 172.21.15.26:0/1668501676 wait complete. 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.665+0000 7f1b7af8d700 1 Processor -- start 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.665+0000 7f1b7af8d700 1 -- start start 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b74152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740b08f0 0x7f1b741590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74154960 con 0x7f1b7415afc0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74153ce0 con 0x7f1b740b08f0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7af8d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b74153fc0 con 0x7f1b7406c5d0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58782/0 (socket says 172.21.15.26:58782) 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 -- 172.21.15.26:0/3325562441 learned_addr learned my addr 172.21.15.26:0/3325562441 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b73fff700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740b08f0 0x7f1b741590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b78d29700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b74152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 -- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 msgr2=0x7f1b74152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b74152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 -- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740b08f0 msgr2=0x7f1b741590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740b08f0 0x7f1b741590d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 -- 172.21.15.26:0/3325562441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b640094a0 con 0x7f1b7415afc0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.666+0000 7f1b7952a700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00cbd0 tx=0x7f1b6c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.667+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00aaa0 con 0x7f1b7415afc0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.667+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c00ac00 con 0x7f1b7415afc0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.667+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c009550 con 0x7f1b7415afc0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.667+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c007070 con 0x7f1b7415afc0 2024-01-16T14:55:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.667+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b74154580 con 0x7f1b7415afc0 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.668+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b74075480 con 0x7f1b7415afc0 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.668+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1b6c0071d0 con 0x7f1b7415afc0 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.669+0000 7f1b717fa700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 0x7f1b5406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.669+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b6c089880 con 0x7f1b7415afc0 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.669+0000 7f1b78d29700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 0x7f1b5406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:22.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.669+0000 7f1b78d29700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 0x7f1b5406d810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009cd0 tx=0x7f1b64009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.674+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b6c054700 con 0x7f1b7415afc0 2024-01-16T14:55:22.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.885+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b74071e10 con 0x7f1b7415afc0 2024-01-16T14:55:22.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.885+0000 7f1b717fa700 1 -- 172.21.15.26:0/3325562441 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b6c017020 con 0x7f1b7415afc0 2024-01-16T14:55:22.869 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 msgr2=0x7f1b5406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 0x7f1b5406d810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009cd0 tx=0x7f1b64009210 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 msgr2=0x7f1b74159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00cbd0 tx=0x7f1b6c00cee0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 shutdown_connections 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f1b5406b350 0x7f1b5406d810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b7415afc0 0x7f1b74159610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1b7406c5d0 0x7f1b74152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 --2- 172.21.15.26:0/3325562441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b740b08f0 0x7f1b741590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 >> 172.21.15.26:0/3325562441 conn(0x7f1b740717f0 msgr2=0x7f1b7406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 shutdown_connections 2024-01-16T14:55:22.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:22.888+0000 7f1b7af8d700 1 -- 172.21.15.26:0/3325562441 wait complete. 2024-01-16T14:55:22.876 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:55:22.878 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:55:22.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T14:55:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.081+0000 7f772563d700 1 Processor -- start 2024-01-16T14:55:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.081+0000 7f772563d700 1 -- start start 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f772563d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77200d2b50 0x7f77200cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f772563d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f77200d0740 0x7f77200d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f772563d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77200ce9a0 con 0x7f772015e080 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f772563d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200d14b0 con 0x7f77200d2b50 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f772563d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77200cec40 con 0x7f77200d0740 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f771e7fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77200d2b50 0x7f77200cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f771e7fc700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77200d2b50 0x7f77200cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58788/0 (socket says 172.21.15.26:58788) 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f771e7fc700 1 -- 172.21.15.26:0/1087220814 learned_addr learned my addr 172.21.15.26:0/1087220814 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.082+0000 7f771f7fe700 1 --2- 172.21.15.26:0/1087220814 >> v2:172.21.15.93:3300/0 conn(0x7f77200d0740 0x7f77200d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771e7fc700 1 -- 172.21.15.26:0/1087220814 >> v2:172.21.15.93:3300/0 conn(0x7f77200d0740 msgr2=0x7f77200d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771e7fc700 1 --2- 172.21.15.26:0/1087220814 >> v2:172.21.15.93:3300/0 conn(0x7f77200d0740 0x7f77200d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771e7fc700 1 -- 172.21.15.26:0/1087220814 >> v1:172.21.15.26:6789/0 conn(0x7f772015e080 legacy=0x7f77200d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771e7fc700 1 -- 172.21.15.26:0/1087220814 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200cef20 con 0x7f77200d2b50 2024-01-16T14:55:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771e7fc700 1 --2- 172.21.15.26:0/1087220814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d2b50 0x7f77200cfa00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f772008e590 tx=0x7f7714009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9490f19c9d8b50ac server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771dffb700 1 -- 172.21.15.26:0/1087220814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400b870 con 0x7f77200d2b50 2024-01-16T14:55:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771dffb700 1 -- 172.21.15.26:0/1087220814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771400b9d0 con 0x7f77200d2b50 2024-01-16T14:55:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f771dffb700 1 -- 172.21.15.26:0/1087220814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400bd60 con 0x7f77200d2b50 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f772563d700 1 -- 172.21.15.26:0/1087220814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d2b50 msgr2=0x7f77200cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.083+0000 7f772563d700 1 --2- 172.21.15.26:0/1087220814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d2b50 0x7f77200cfa00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f772008e590 tx=0x7f7714009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 -- 172.21.15.26:0/1087220814 shutdown_connections 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 --2- 172.21.15.26:0/1087220814 >> v2:172.21.15.93:3300/0 conn(0x7f77200d0740 0x7f77200d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 --2- 172.21.15.26:0/1087220814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d2b50 0x7f77200cfa00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 -- 172.21.15.26:0/1087220814 >> 172.21.15.26:0/1087220814 conn(0x7f772009d180 msgr2=0x7f77200d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 -- 172.21.15.26:0/1087220814 shutdown_connections 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.084+0000 7f772563d700 1 -- 172.21.15.26:0/1087220814 wait complete. 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.085+0000 7f772563d700 1 Processor -- start 2024-01-16T14:55:23.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.085+0000 7f772563d700 1 -- start start 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 0x7f7720160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f772015e080 0x7f772015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772015be40 con 0x7f77200d0740 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77200ca6b0 con 0x7f77200d2b50 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f772563d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77200ca950 con 0x7f772015e080 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f771effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f771effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58800/0 (socket says 172.21.15.26:58800) 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f771effd700 1 -- 172.21.15.26:0/4184169269 learned_addr learned my addr 172.21.15.26:0/4184169269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f771e7fc700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 0x7f7720160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.086+0000 7f771f7fe700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f772015e080 0x7f772015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 -- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f772015e080 msgr2=0x7f772015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f772015e080 0x7f772015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 -- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 msgr2=0x7f7720160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 0x7f7720160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 -- 172.21.15.26:0/4184169269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714009550 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771e7fc700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 0x7f7720160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f771effd700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f771000c450 tx=0x7f771000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100052d0 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710005430 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100038c0 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77200cac30 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.087+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77200cb2e0 con 0x7f77200d0740 2024-01-16T14:55:23.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.088+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7720075480 con 0x7f77200d0740 2024-01-16T14:55:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.089+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7710003a20 con 0x7f77200d0740 2024-01-16T14:55:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.089+0000 7f77077fe700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 0x7f770c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.089+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7710088ef0 con 0x7f77200d0740 2024-01-16T14:55:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.089+0000 7f771e7fc700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 0x7f770c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.090+0000 7f771e7fc700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 0x7f770c06d810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77200d2f70 tx=0x7f7714008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.094+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7710053d60 con 0x7f77200d0740 2024-01-16T14:55:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.323+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f772006ca50 con 0x7f77200d0740 2024-01-16T14:55:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.323+0000 7f77077fe700 1 -- 172.21.15.26:0/4184169269 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7710057380 con 0x7f77200d0740 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 msgr2=0x7f770c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 0x7f770c06d810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77200d2f70 tx=0x7f7714008030 comp rx=0 tx=0).stop 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 msgr2=0x7f77200b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f771000c450 tx=0x7f771000c810 comp rx=0 tx=0).stop 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 shutdown_connections 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f772015e080 0x7f772015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f770c06b350 0x7f770c06d810 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77200d0740 0x7f77200b4550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 --2- 172.21.15.26:0/4184169269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f77200d2b50 0x7f7720160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 >> 172.21.15.26:0/4184169269 conn(0x7f772009d180 msgr2=0x7f77200cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 shutdown_connections 2024-01-16T14:55:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.326+0000 7f772563d700 1 -- 172.21.15.26:0/4184169269 wait complete. 2024-01-16T14:55:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-16T14:55:23.323 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-01-16T14:55:23.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:23.324 DEBUG:teuthology.orchestra.run.smithi093:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-01-16T14:55:23.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:55:23.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:23.370 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-16T14:55:23.405 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:23.406 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:23.427 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:23.427 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:23.489 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:23.490 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:23.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:23.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.754+0000 7f05831eb700 1 Processor -- start 2024-01-16T14:55:23.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.754+0000 7f05831eb700 1 -- start start 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f05831eb700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f057c1445f0 0x7f057c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f05831eb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f057c06cae0 0x7f057c0726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f05831eb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057c1449d0 con 0x7f057c06e480 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f05831eb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c071650 con 0x7f057c06cae0 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f05831eb700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c071930 con 0x7f057c1445f0 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f057bfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f057c06cae0 0x7f057c0726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.755+0000 7f0581788700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f057c06e480 0x7f057c0cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42280/0 (socket says 172.21.15.26:42280) 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f0581788700 1 -- 172.21.15.26:0/3426558025 learned_addr learned my addr 172.21.15.26:0/3426558025 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f0580f87700 1 --2- 172.21.15.26:0/3426558025 >> v2:172.21.15.93:3300/0 conn(0x7f057c1445f0 0x7f057c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057bfff700 1 -- 172.21.15.26:0/3426558025 >> v2:172.21.15.93:3300/0 conn(0x7f057c1445f0 msgr2=0x7f057c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057bfff700 1 --2- 172.21.15.26:0/3426558025 >> v2:172.21.15.93:3300/0 conn(0x7f057c1445f0 0x7f057c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057bfff700 1 -- 172.21.15.26:0/3426558025 >> v1:172.21.15.26:6789/0 conn(0x7f057c06e480 legacy=0x7f057c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057bfff700 1 -- 172.21.15.26:0/3426558025 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c071c10 con 0x7f057c06cae0 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057b7fe700 1 -- 172.21.15.26:0/3426558025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618466769 0 0) 0x7f057c1449d0 con 0x7f057c06e480 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057bfff700 1 --2- 172.21.15.26:0/3426558025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06cae0 0x7f057c0726f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f057c0cd480 tx=0x7f0568009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59da3822a5c49b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057b7fe700 1 -- 172.21.15.26:0/3426558025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011830 con 0x7f057c06cae0 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057b7fe700 1 -- 172.21.15.26:0/3426558025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568011990 con 0x7f057c06cae0 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.756+0000 7f057b7fe700 1 -- 172.21.15.26:0/3426558025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568011d20 con 0x7f057c06cae0 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 -- 172.21.15.26:0/3426558025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06cae0 msgr2=0x7f057c0726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 --2- 172.21.15.26:0/3426558025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06cae0 0x7f057c0726f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f057c0cd480 tx=0x7f0568009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 -- 172.21.15.26:0/3426558025 shutdown_connections 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 --2- 172.21.15.26:0/3426558025 >> v2:172.21.15.93:3300/0 conn(0x7f057c1445f0 0x7f057c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 --2- 172.21.15.26:0/3426558025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06cae0 0x7f057c0726f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 -- 172.21.15.26:0/3426558025 >> 172.21.15.26:0/3426558025 conn(0x7f057c0cf980 msgr2=0x7f057c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:23.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 -- 172.21.15.26:0/3426558025 shutdown_connections 2024-01-16T14:55:23.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.757+0000 7f05831eb700 1 -- 172.21.15.26:0/3426558025 wait complete. 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.758+0000 7f05831eb700 1 Processor -- start 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.758+0000 7f05831eb700 1 -- start start 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f057c06cae0 0x7f057c137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f057c1445f0 0x7f057c13f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0c5670 con 0x7f057c06e480 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0c4a60 con 0x7f057c06cae0 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f05831eb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0c4d40 con 0x7f057c1445f0 2024-01-16T14:55:23.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.759+0000 7f057bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58818/0 (socket says 172.21.15.26:58818) 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 -- 172.21.15.26:0/404678647 learned_addr learned my addr 172.21.15.26:0/404678647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f0580f87700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f057c06cae0 0x7f057c137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f0581788700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f057c1445f0 0x7f057c13f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 -- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f057c1445f0 msgr2=0x7f057c13f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f057c1445f0 0x7f057c13f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 -- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f057c06cae0 msgr2=0x7f057c137540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f057c06cae0 0x7f057c137540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 -- 172.21.15.26:0/404678647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05680094a0 con 0x7f057c06e480 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f057bfff700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05680090b0 tx=0x7f0568004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680036a0 con 0x7f057c06e480 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056801f070 con 0x7f057c06e480 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c0c5020 con 0x7f057c06e480 2024-01-16T14:55:23.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568003a60 con 0x7f057c06e480 2024-01-16T14:55:23.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.760+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c0b4940 con 0x7f057c06e480 2024-01-16T14:55:23.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.761+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c13e140 con 0x7f057c06e480 2024-01-16T14:55:23.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.762+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0568003bc0 con 0x7f057c06e480 2024-01-16T14:55:23.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.762+0000 7f05797fa700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 0x7f056006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:23.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.762+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0568021070 con 0x7f057c06e480 2024-01-16T14:55:23.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.762+0000 7f0580f87700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 0x7f056006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:23.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.763+0000 7f0580f87700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 0x7f056006d810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0570006b20 tx=0x7f0570008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:23.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.767+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0568056280 con 0x7f057c06e480 2024-01-16T14:55:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.976+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f057c075480 con 0x7f057c06e480 2024-01-16T14:55:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.978+0000 7f05797fa700 1 -- 172.21.15.26:0/404678647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f05680598a0 con 0x7f057c06e480 2024-01-16T14:55:23.962 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:23.962 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 msgr2=0x7f056006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 0x7f056006d810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0570006b20 tx=0x7f0570008040 comp rx=0 tx=0).stop 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 msgr2=0x7f057c13f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05680090b0 tx=0x7f0568004750 comp rx=0 tx=0).stop 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 shutdown_connections 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f057c1445f0 0x7f057c13f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f057c06cae0 0x7f057c137540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f056006b350 0x7f056006d810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 --2- 172.21.15.26:0/404678647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f057c06e480 0x7f057c13f310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 >> 172.21.15.26:0/404678647 conn(0x7f057c0cf980 msgr2=0x7f057c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 shutdown_connections 2024-01-16T14:55:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:23.980+0000 7f05831eb700 1 -- 172.21.15.26:0/404678647 wait complete. 2024-01-16T14:55:23.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:23.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.134+0000 7fec09b56700 1 Processor -- start 2024-01-16T14:55:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.134+0000 7fec09b56700 1 -- start start 2024-01-16T14:55:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec09b56700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec0415be40 0x7fec040af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec09b56700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fec040d0ec0 0x7fec040b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec09b56700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec040af940 con 0x7fec040b1a90 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec09b56700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec040b1130 con 0x7fec0415be40 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec09b56700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec04067420 con 0x7fec040d0ec0 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec037fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec0415be40 0x7fec040af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec037fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec0415be40 0x7fec040af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58824/0 (socket says 172.21.15.26:58824) 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.135+0000 7fec037fe700 1 -- 172.21.15.26:0/817842020 learned_addr learned my addr 172.21.15.26:0/817842020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec02ffd700 1 --2- 172.21.15.26:0/817842020 >> v2:172.21.15.93:3300/0 conn(0x7fec040d0ec0 0x7fec040b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec037fe700 1 -- 172.21.15.26:0/817842020 >> v2:172.21.15.93:3300/0 conn(0x7fec040d0ec0 msgr2=0x7fec040b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec037fe700 1 --2- 172.21.15.26:0/817842020 >> v2:172.21.15.93:3300/0 conn(0x7fec040d0ec0 0x7fec040b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec037fe700 1 -- 172.21.15.26:0/817842020 >> v1:172.21.15.93:6789/0 conn(0x7fec040b1a90 legacy=0x7fec0406b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec037fe700 1 -- 172.21.15.26:0/817842020 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec040b0ab0 con 0x7fec0415be40 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec037fe700 1 --2- 172.21.15.26:0/817842020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec040af400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec040709e0 tx=0x7febf4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2964d0a90deb799c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec027fc700 1 -- 172.21.15.26:0/817842020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4004230 con 0x7fec0415be40 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec027fc700 1 -- 172.21.15.26:0/817842020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf4004390 con 0x7fec0415be40 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.136+0000 7fec027fc700 1 -- 172.21.15.26:0/817842020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf40036a0 con 0x7fec0415be40 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 -- 172.21.15.26:0/817842020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 msgr2=0x7fec040af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 --2- 172.21.15.26:0/817842020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec040af400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec040709e0 tx=0x7febf4009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 -- 172.21.15.26:0/817842020 shutdown_connections 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 --2- 172.21.15.26:0/817842020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec040af400 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 --2- 172.21.15.26:0/817842020 >> v2:172.21.15.93:3300/0 conn(0x7fec040d0ec0 0x7fec040b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 -- 172.21.15.26:0/817842020 >> 172.21.15.26:0/817842020 conn(0x7fec040717f0 msgr2=0x7fec040b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 -- 172.21.15.26:0/817842020 shutdown_connections 2024-01-16T14:55:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.137+0000 7fec09b56700 1 -- 172.21.15.26:0/817842020 wait complete. 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.138+0000 7fec09b56700 1 Processor -- start 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.138+0000 7fec09b56700 1 -- start start 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec040d0ec0 0x7fec0414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec040ae330 0x7fec0414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040adea0 con 0x7fec0415be40 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040ad290 con 0x7fec040ae330 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec09b56700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec040ad570 con 0x7fec040d0ec0 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec02ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec02ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58828/0 (socket says 172.21.15.26:58828) 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec02ffd700 1 -- 172.21.15.26:0/534101084 learned_addr learned my addr 172.21.15.26:0/534101084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec037fe700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec040d0ec0 0x7fec0414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.139+0000 7fec03fff700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec040ae330 0x7fec0414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 -- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec040d0ec0 msgr2=0x7fec0414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec040d0ec0 0x7fec0414f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 -- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec040ae330 msgr2=0x7fec0414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec040ae330 0x7fec0414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 -- 172.21.15.26:0/534101084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4009550 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec02ffd700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febf800e810 tx=0x7febf800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800c070 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8013610 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80139c0 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec040ad850 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.140+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec04156ff0 con 0x7fec0415be40 2024-01-16T14:55:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.141+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec040ae010 con 0x7fec0415be40 2024-01-16T14:55:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.142+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7febf8013b20 con 0x7fec0415be40 2024-01-16T14:55:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.142+0000 7febebfff700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 0x7febf006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.142+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febf80898f0 con 0x7fec0415be40 2024-01-16T14:55:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.142+0000 7fec037fe700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 0x7febf006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.143+0000 7fec037fe700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 0x7febf006d810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febf4009130 tx=0x7febf4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.147+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febf8054770 con 0x7fec0415be40 2024-01-16T14:55:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.357+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec04075480 con 0x7fec0415be40 2024-01-16T14:55:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.357+0000 7febebfff700 1 -- 172.21.15.26:0/534101084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7febf8057d90 con 0x7fec0415be40 2024-01-16T14:55:24.341 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.359+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 msgr2=0x7febf006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.359+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 0x7febf006d810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febf4009130 tx=0x7febf4008030 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 msgr2=0x7fec0414f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febf800e810 tx=0x7febf800eb20 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 shutdown_connections 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec040ae330 0x7fec0414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec040d0ec0 0x7fec0414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7febf006b350 0x7febf006d810 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 --2- 172.21.15.26:0/534101084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0415be40 0x7fec0414f690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 >> 172.21.15.26:0/534101084 conn(0x7fec040717f0 msgr2=0x7fec040cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:24.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 shutdown_connections 2024-01-16T14:55:24.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.360+0000 7fec09b56700 1 -- 172.21.15.26:0/534101084 wait complete. 2024-01-16T14:55:24.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:24.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.513+0000 7f5d4bc98700 1 Processor -- start 2024-01-16T14:55:24.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.513+0000 7f5d4bc98700 1 -- start start 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d4bc98700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b3e20 0x7f5d44162900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d4bc98700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d440b57b0 0x7f5d441665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d4bc98700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d44162e40 con 0x7f5d44166b30 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d4bc98700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d44165ef0 con 0x7f5d440b57b0 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d4bc98700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44163070 con 0x7f5d440b3e20 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d440b57b0 0x7f5d441665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d440b57b0 0x7f5d441665f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50110/0 (socket says 172.21.15.26:50110) 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 -- 172.21.15.26:0/749142110 learned_addr learned my addr 172.21.15.26:0/749142110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49a34700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b3e20 0x7f5d44162900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 -- 172.21.15.26:0/749142110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b3e20 msgr2=0x7f5d44162900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b3e20 0x7f5d44162900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 -- 172.21.15.26:0/749142110 >> v1:172.21.15.93:6789/0 conn(0x7f5d44166b30 legacy=0x7f5d4406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d48a32700 1 -- 172.21.15.26:0/749142110 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012494553 0 0) 0x7f5d44162e40 con 0x7f5d44166b30 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 -- 172.21.15.26:0/749142110 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44163350 con 0x7f5d440b57b0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.514+0000 7f5d49233700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b57b0 0x7f5d441665f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098f0 tx=0x7f5d40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdd4461eff76072 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d48a32700 1 -- 172.21.15.26:0/749142110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40011830 con 0x7f5d440b57b0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d48a32700 1 -- 172.21.15.26:0/749142110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40011990 con 0x7f5d440b57b0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d48a32700 1 -- 172.21.15.26:0/749142110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40011d20 con 0x7f5d440b57b0 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 -- 172.21.15.26:0/749142110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b57b0 msgr2=0x7f5d441665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b57b0 0x7f5d441665f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5d400098f0 tx=0x7f5d40009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 -- 172.21.15.26:0/749142110 shutdown_connections 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b3e20 0x7f5d44162900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/749142110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b57b0 0x7f5d441665f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 -- 172.21.15.26:0/749142110 >> 172.21.15.26:0/749142110 conn(0x7f5d440d0aa0 msgr2=0x7f5d440d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 -- 172.21.15.26:0/749142110 shutdown_connections 2024-01-16T14:55:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.515+0000 7f5d4bc98700 1 -- 172.21.15.26:0/749142110 wait complete. 2024-01-16T14:55:24.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 Processor -- start 2024-01-16T14:55:24.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 -- start start 2024-01-16T14:55:24.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 0x7f5d4406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5d44140850 0x7f5d4406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4413fde0 con 0x7f5d440b3e20 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4413ed40 con 0x7f5d44140850 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.517+0000 7f5d4bc98700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4413f020 con 0x7f5d440b57b0 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50122/0 (socket says 172.21.15.26:50122) 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 -- 172.21.15.26:0/3523616303 learned_addr learned my addr 172.21.15.26:0/3523616303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d4a235700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5d44140850 0x7f5d4406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49233700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 0x7f5d4406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 -- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 msgr2=0x7f5d4406e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 0x7f5d4406e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 -- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5d44140850 msgr2=0x7f5d4406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5d44140850 0x7f5d4406ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 -- 172.21.15.26:0/3523616303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40009550 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49233700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 0x7f5d4406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d49a34700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e850 tx=0x7f5d3800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380095b0 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800ee50 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4413f300 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800ca70 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.518+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4413f8c0 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.519+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d44075480 con 0x7f5d440b3e20 2024-01-16T14:55:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.520+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5d3800cbd0 con 0x7f5d440b3e20 2024-01-16T14:55:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.520+0000 7f5d367fc700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 0x7f5d2806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.521+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d38089cd0 con 0x7f5d440b3e20 2024-01-16T14:55:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.521+0000 7f5d49233700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 0x7f5d2806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.521+0000 7f5d49233700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 0x7f5d2806d810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d4413d5c0 tx=0x7f5d40008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.525+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d38054a20 con 0x7f5d440b3e20 2024-01-16T14:55:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.738+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d4413fab0 con 0x7f5d440b3e20 2024-01-16T14:55:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.739+0000 7f5d367fc700 1 -- 172.21.15.26:0/3523616303 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d38058040 con 0x7f5d440b3e20 2024-01-16T14:55:24.723 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 msgr2=0x7f5d2806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 0x7f5d2806d810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d4413d5c0 tx=0x7f5d40008030 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 msgr2=0x7f5d4413e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e850 tx=0x7f5d3800ec10 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 shutdown_connections 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5d44140850 0x7f5d4406ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5d2806b350 0x7f5d2806d810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d440b3e20 0x7f5d4413e3d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 --2- 172.21.15.26:0/3523616303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5d440b57b0 0x7f5d4406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.741+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 >> 172.21.15.26:0/3523616303 conn(0x7f5d440d0aa0 msgr2=0x7f5d4406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.742+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 shutdown_connections 2024-01-16T14:55:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.742+0000 7f5d4bc98700 1 -- 172.21.15.26:0/3523616303 wait complete. 2024-01-16T14:55:24.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.896+0000 7f57a00cb700 1 Processor -- start 2024-01-16T14:55:24.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.897+0000 7f57a00cb700 1 -- start start 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f57a00cb700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57980b3270 0x7f579806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f57a00cb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980d12b0 con 0x7f579815e200 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f57a00cb700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57980d0980 con 0x7f579806c6e0 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f57a00cb700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57980d0c60 con 0x7f57980b3270 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579de67700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f579815e200 0x7f579806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40970/0 (socket says 172.21.15.26:40970) 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579de67700 1 -- 172.21.15.26:0/1021949051 learned_addr learned my addr 172.21.15.26:0/1021949051 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:24.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579e668700 1 --2- 172.21.15.26:0/1021949051 >> v2:172.21.15.26:3300/0 conn(0x7f57980b3270 0x7f579806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249605197 0 0) 0x7f57980d12b0 con 0x7f579815e200 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57840035b0 con 0x7f579815e200 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391914211 0 0) 0x7f57980d0980 con 0x7f579806c6e0 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57980d12b0 con 0x7f579806c6e0 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579e668700 1 -- 172.21.15.26:0/1021949051 >> v1:172.21.15.93:6789/0 conn(0x7f579806c6e0 legacy=0x7f57980d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579e668700 1 -- 172.21.15.26:0/1021949051 >> v1:172.21.15.26:6789/0 conn(0x7f579815e200 legacy=0x7f579806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.898+0000 7f579e668700 1 -- 172.21.15.26:0/1021949051 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980d0f40 con 0x7f57980b3270 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f579e668700 1 --2- 172.21.15.26:0/1021949051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57980b3270 0x7f579806b9a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d90 tx=0x7f579000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819055874f7f1e77 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000e070 con 0x7f57980b3270 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000bb20 con 0x7f57980b3270 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f579ce65700 1 -- 172.21.15.26:0/1021949051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790012430 con 0x7f57980b3270 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 -- 172.21.15.26:0/1021949051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57980b3270 msgr2=0x7f579806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 --2- 172.21.15.26:0/1021949051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57980b3270 0x7f579806b9a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5790009d90 tx=0x7f579000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:24.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 -- 172.21.15.26:0/1021949051 shutdown_connections 2024-01-16T14:55:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 --2- 172.21.15.26:0/1021949051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57980b3270 0x7f579806b9a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 -- 172.21.15.26:0/1021949051 >> 172.21.15.26:0/1021949051 conn(0x7f579809d180 msgr2=0x7f57980d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 -- 172.21.15.26:0/1021949051 shutdown_connections 2024-01-16T14:55:24.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.899+0000 7f57a00cb700 1 -- 172.21.15.26:0/1021949051 wait complete. 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 Processor -- start 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 -- start start 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 0x7f57980cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f579815e200 0x7f57980caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b8b50 con 0x7f579806c6e0 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b7ed0 con 0x7f57980b3270 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.901+0000 7f57a00cb700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b81b0 con 0x7f579815e200 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50132/0 (socket says 172.21.15.26:50132) 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 -- 172.21.15.26:0/841191305 learned_addr learned my addr 172.21.15.26:0/841191305 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579d666700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 0x7f57980cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579e668700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f579815e200 0x7f57980caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 -- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f579815e200 msgr2=0x7f57980caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f579815e200 0x7f57980caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 -- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 msgr2=0x7f57980cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 0x7f57980cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 -- 172.21.15.26:0/841191305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5790009a70 con 0x7f579806c6e0 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579d666700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 0x7f57980cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:24.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f579de67700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5794001eb0 tx=0x7f579400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794005d70 con 0x7f579806c6e0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940070d0 con 0x7f579806c6e0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794007460 con 0x7f579806c6e0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57980b8490 con 0x7f579806c6e0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.902+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57980b94b0 con 0x7f579806c6e0 2024-01-16T14:55:24.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.903+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5798075480 con 0x7f579806c6e0 2024-01-16T14:55:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.904+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f57940075c0 con 0x7f579806c6e0 2024-01-16T14:55:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.904+0000 7f578e7fc700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 0x7f577c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.905+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5794089100 con 0x7f579806c6e0 2024-01-16T14:55:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.905+0000 7f579d666700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 0x7f577c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:24.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.905+0000 7f579d666700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 0x7f577c06d810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57980d34c0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:24.909+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5794053f60 con 0x7f579806c6e0 2024-01-16T14:55:25.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.117+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57981e6ef0 con 0x7f579806c6e0 2024-01-16T14:55:25.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.118+0000 7f578e7fc700 1 -- 172.21.15.26:0/841191305 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5794057580 con 0x7f579806c6e0 2024-01-16T14:55:25.102 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 msgr2=0x7f577c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 0x7f577c06d810 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57980d34c0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2024-01-16T14:55:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 msgr2=0x7f57980b8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5794001eb0 tx=0x7f579400c860 comp rx=0 tx=0).stop 2024-01-16T14:55:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 shutdown_connections 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f579815e200 0x7f57980caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f577c06b350 0x7f577c06d810 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f579806c6e0 0x7f57980b8f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 --2- 172.21.15.26:0/841191305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57980b3270 0x7f57980cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 >> 172.21.15.26:0/841191305 conn(0x7f579809d180 msgr2=0x7f5798160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 shutdown_connections 2024-01-16T14:55:25.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.120+0000 7f57a00cb700 1 -- 172.21.15.26:0/841191305 wait complete. 2024-01-16T14:55:25.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T14:55:25.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.271+0000 7f503f6ad700 1 Processor -- start 2024-01-16T14:55:25.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.271+0000 7f503f6ad700 1 -- start start 2024-01-16T14:55:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503f6ad700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503f6ad700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50380cf2b0 0x7f50380d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503f6ad700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380d0a90 con 0x7f50380ce5c0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503f6ad700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503806ee10 con 0x7f50380cf2b0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503f6ad700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503806f0f0 con 0x7f503815bde0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503cc48700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50380cf2b0 0x7f50380d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503dc4a700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50380ce5c0 0x7f50380cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40984/0 (socket says 172.21.15.26:40984) 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503cc48700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50380cf2b0 0x7f50380d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50140/0 (socket says 172.21.15.26:50140) 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.272+0000 7f503dc4a700 1 -- 172.21.15.26:0/2188597762 learned_addr learned my addr 172.21.15.26:0/2188597762 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503d449700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503cc48700 1 -- 172.21.15.26:0/2188597762 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 msgr2=0x7f503815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503cc48700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503cc48700 1 -- 172.21.15.26:0/2188597762 >> v1:172.21.15.26:6789/0 conn(0x7f50380ce5c0 legacy=0x7f50380cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503cc48700 1 -- 172.21.15.26:0/2188597762 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503806f3d0 con 0x7f50380cf2b0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f503cc48700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f50380d01b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5028009840 tx=0x7f5028009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3340609342081d02 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f502ffff700 1 -- 172.21.15.26:0/2188597762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028011830 con 0x7f50380cf2b0 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f502ffff700 1 -- 172.21.15.26:0/2188597762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5028011990 con 0x7f50380cf2b0 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.273+0000 7f502ffff700 1 -- 172.21.15.26:0/2188597762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028011d20 con 0x7f50380cf2b0 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 -- 172.21.15.26:0/2188597762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 msgr2=0x7f50380d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f50380d01b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5028009840 tx=0x7f5028009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 -- 172.21.15.26:0/2188597762 shutdown_connections 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 --2- 172.21.15.26:0/2188597762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f50380d01b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 -- 172.21.15.26:0/2188597762 >> 172.21.15.26:0/2188597762 conn(0x7f50380b3720 msgr2=0x7f50380b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 -- 172.21.15.26:0/2188597762 shutdown_connections 2024-01-16T14:55:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.274+0000 7f503f6ad700 1 -- 172.21.15.26:0/2188597762 wait complete. 2024-01-16T14:55:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.275+0000 7f503f6ad700 1 Processor -- start 2024-01-16T14:55:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.276+0000 7f503f6ad700 1 -- start start 2024-01-16T14:55:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.276+0000 7f503f6ad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50380ce5c0 0x7f503814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503f6ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503f6ad700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503f6ad700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380c89e0 con 0x7f50380cf2b0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503f6ad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503814df20 con 0x7f50380ce5c0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503f6ad700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503814ac50 con 0x7f503815bde0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50152/0 (socket says 172.21.15.26:50152) 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 -- 172.21.15.26:0/3816634255 learned_addr learned my addr 172.21.15.26:0/3816634255 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503d449700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50380ce5c0 0x7f503814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503dc4a700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 -- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 msgr2=0x7f503806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503806ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 -- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50380ce5c0 msgr2=0x7f503814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50380ce5c0 0x7f503814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 -- 172.21.15.26:0/3816634255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280094a0 con 0x7f50380cf2b0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f503cc48700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5028009b90 tx=0x7f5028004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50280036a0 con 0x7f50380cf2b0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.277+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502801f070 con 0x7f50380cf2b0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.278+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503814b480 con 0x7f50380cf2b0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.278+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028003a60 con 0x7f50380cf2b0 2024-01-16T14:55:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.278+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50380c5a30 con 0x7f50380cf2b0 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.279+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380c5890 con 0x7f50380cf2b0 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.279+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5028003bc0 con 0x7f50380cf2b0 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.280+0000 7f502dffb700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 0x7f501c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.280+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5028021070 con 0x7f50380cf2b0 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.280+0000 7f503d449700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 0x7f501c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.280+0000 7f503d449700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 0x7f501c06d810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5034006b20 tx=0x7f5034008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.284+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50380c5890 con 0x7f50380cf2b0 2024-01-16T14:55:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.514+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50380c5890 con 0x7f50380cf2b0 2024-01-16T14:55:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.514+0000 7f502dffb700 1 -- 172.21.15.26:0/3816634255 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f50380c5890 con 0x7f50380cf2b0 2024-01-16T14:55:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 msgr2=0x7f501c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 0x7f501c06d810 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5034006b20 tx=0x7f5034008040 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 msgr2=0x7f503814a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5028009b90 tx=0x7f5028004750 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 shutdown_connections 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f503815bde0 0x7f503806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50380ce5c0 0x7f503814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f501c06b350 0x7f501c06d810 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 --2- 172.21.15.26:0/3816634255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cf2b0 0x7f503814a710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 >> 172.21.15.26:0/3816634255 conn(0x7f50380b3720 msgr2=0x7f50380cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 shutdown_connections 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.516+0000 7f503f6ad700 1 -- 172.21.15.26:0/3816634255 wait complete. 2024-01-16T14:55:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-16T14:55:25.514 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-01-16T14:55:25.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:25.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-01-16T14:55:25.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-16T14:55:25.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:25.563 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-16T14:55:25.582 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:25.582 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:25.604 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:25.604 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:25.666 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:25.666 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:25.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.889+0000 7fb746a29700 1 Processor -- start 2024-01-16T14:55:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.889+0000 7fb746a29700 1 -- start start 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb746a29700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb74006f990 0x7fb740165b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb746a29700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740144c90 con 0x7fb7401448b0 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb746a29700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400b4c80 con 0x7fb74006ec70 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb746a29700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400b4f60 con 0x7fb74006f990 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb74006f990 0x7fb740165b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb74006f990 0x7fb740165b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50154/0 (socket says 172.21.15.26:50154) 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb73ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb7401448b0 0x7fb7400b44c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40992/0 (socket says 172.21.15.26:40992) 2024-01-16T14:55:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 -- 172.21.15.26:0/3320309174 learned_addr learned my addr 172.21.15.26:0/3320309174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 -- 172.21.15.26:0/3320309174 >> v1:172.21.15.93:6789/0 conn(0x7fb74006ec70 legacy=0x7fb74006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 -- 172.21.15.26:0/3320309174 >> v1:172.21.15.26:6789/0 conn(0x7fb7401448b0 legacy=0x7fb7400b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 -- 172.21.15.26:0/3320309174 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400b5240 con 0x7fb74006f990 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb744fc6700 1 --2- 172.21.15.26:0/3320309174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb740165b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb738009e50 tx=0x7fb73800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20d67b6f3f5b5ca server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.890+0000 7fb73effd700 1 -- 172.21.15.26:0/3320309174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800e070 con 0x7fb74006f990 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb73effd700 1 -- 172.21.15.26:0/3320309174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73800b9c0 con 0x7fb74006f990 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb73effd700 1 -- 172.21.15.26:0/3320309174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73800bd70 con 0x7fb74006f990 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 -- 172.21.15.26:0/3320309174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 msgr2=0x7fb740165b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 --2- 172.21.15.26:0/3320309174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb740165b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb738009e50 tx=0x7fb73800b330 comp rx=0 tx=0).stop 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 -- 172.21.15.26:0/3320309174 shutdown_connections 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 --2- 172.21.15.26:0/3320309174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb740165b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 -- 172.21.15.26:0/3320309174 >> 172.21.15.26:0/3320309174 conn(0x7fb7400d0bc0 msgr2=0x7fb7400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 -- 172.21.15.26:0/3320309174 shutdown_connections 2024-01-16T14:55:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.891+0000 7fb746a29700 1 -- 172.21.15.26:0/3320309174 wait complete. 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.893+0000 7fb746a29700 1 Processor -- start 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.893+0000 7fb746a29700 1 -- start start 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb74006ec70 0x7fb74015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7401448b0 0x7fb7401e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400aefe0 con 0x7fb74006f990 2024-01-16T14:55:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400af2c0 con 0x7fb74006ec70 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400afcd0 con 0x7fb7401448b0 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50158/0 (socket says 172.21.15.26:50158) 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 -- 172.21.15.26:0/4071062279 learned_addr learned my addr 172.21.15.26:0/4071062279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73ffff700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb74006ec70 0x7fb74015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb744fc6700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7401448b0 0x7fb7401e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 -- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7401448b0 msgr2=0x7fb7401e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7401448b0 0x7fb7401e4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 -- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb74006ec70 msgr2=0x7fb74015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb74006ec70 0x7fb74015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 -- 172.21.15.26:0/4071062279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738009b30 con 0x7fb74006f990 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73f7fe700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c700 tx=0x7fb73400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7340052d0 con 0x7fb74006f990 2024-01-16T14:55:25.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734005430 con 0x7fb74006f990 2024-01-16T14:55:25.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7400affb0 con 0x7fb74006f990 2024-01-16T14:55:25.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.894+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734003880 con 0x7fb74006f990 2024-01-16T14:55:25.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.895+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74013e670 con 0x7fb74006f990 2024-01-16T14:55:25.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.895+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400b04d0 con 0x7fb74006f990 2024-01-16T14:55:25.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.896+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb7340039e0 con 0x7fb74006f990 2024-01-16T14:55:25.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.896+0000 7fb73cff9700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 0x7fb72c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:25.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.897+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb734088ea0 con 0x7fb74006f990 2024-01-16T14:55:25.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.897+0000 7fb73ffff700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 0x7fb72c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:25.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.897+0000 7fb73ffff700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 0x7fb72c06d810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb730000f60 tx=0x7fb730007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:25.901+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7400b04d0 con 0x7fb74006f990 2024-01-16T14:55:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.112+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7400b04d0 con 0x7fb74006f990 2024-01-16T14:55:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.113+0000 7fb73cff9700 1 -- 172.21.15.26:0/4071062279 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb7400b04d0 con 0x7fb74006f990 2024-01-16T14:55:26.098 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:26.098 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 msgr2=0x7fb72c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 0x7fb72c06d810 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb730000f60 tx=0x7fb730007ed0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 msgr2=0x7fb7401e4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb73400c700 tx=0x7fb73400ca10 comp rx=0 tx=0).stop 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 shutdown_connections 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7401448b0 0x7fb7401e4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb74006ec70 0x7fb74015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb72c06b350 0x7fb72c06d810 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 --2- 172.21.15.26:0/4071062279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb74006f990 0x7fb7401e4110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 >> 172.21.15.26:0/4071062279 conn(0x7fb7400d0bc0 msgr2=0x7fb7400d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 shutdown_connections 2024-01-16T14:55:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.117+0000 7fb746a29700 1 -- 172.21.15.26:0/4071062279 wait complete. 2024-01-16T14:55:26.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:26.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:26.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 Processor -- start 2024-01-16T14:55:26.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 -- start start 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f706c0b8870 0x7f706c0d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0b73d0 con 0x7f706c15e210 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0b6df0 con 0x7f706c0d0ee0 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.271+0000 7f7071e59700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c15e5f0 con 0x7f706c0b8870 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f706c15e210 0x7f706c0d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41008/0 (socket says 172.21.15.26:41008) 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706b7fe700 1 -- 172.21.15.26:0/466329360 learned_addr learned my addr 172.21.15.26:0/466329360 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706bfff700 1 --2- 172.21.15.26:0/466329360 >> v2:172.21.15.93:3300/0 conn(0x7f706c0b8870 0x7f706c0d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724304317 0 0) 0x7f706c0b73d0 con 0x7f706c15e210 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70540035b0 con 0x7f706c15e210 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706bfff700 1 -- 172.21.15.26:0/466329360 >> v1:172.21.15.93:6789/0 conn(0x7f706c0d0ee0 legacy=0x7f706c0d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144336004 0 0) 0x7f706c0b6df0 con 0x7f706c0d0ee0 2024-01-16T14:55:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706bfff700 1 -- 172.21.15.26:0/466329360 >> v1:172.21.15.26:6789/0 conn(0x7f706c15e210 legacy=0x7f706c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706bfff700 1 -- 172.21.15.26:0/466329360 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0cfae0 con 0x7f706c0b8870 2024-01-16T14:55:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.272+0000 7f706bfff700 1 --2- 172.21.15.26:0/466329360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0b8870 0x7f706c0d21e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7064009d90 tx=0x7f706400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a80eea2cb6272ff0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400e070 con 0x7f706c0b8870 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706400bb20 con 0x7f706c0b8870 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f706a7fc700 1 -- 172.21.15.26:0/466329360 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400bed0 con 0x7f706c0b8870 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 -- 172.21.15.26:0/466329360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0b8870 msgr2=0x7f706c0d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 --2- 172.21.15.26:0/466329360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0b8870 0x7f706c0d21e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7064009d90 tx=0x7f706400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 -- 172.21.15.26:0/466329360 shutdown_connections 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 --2- 172.21.15.26:0/466329360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0b8870 0x7f706c0d21e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 -- 172.21.15.26:0/466329360 >> 172.21.15.26:0/466329360 conn(0x7f706c09d180 msgr2=0x7f706c0d3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 -- 172.21.15.26:0/466329360 shutdown_connections 2024-01-16T14:55:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.273+0000 7f7071e59700 1 -- 172.21.15.26:0/466329360 wait complete. 2024-01-16T14:55:26.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.275+0000 7f7071e59700 1 Processor -- start 2024-01-16T14:55:26.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.275+0000 7f7071e59700 1 -- start start 2024-01-16T14:55:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 0x7f706c09f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 0x7f706c09fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0a0620 con 0x7f706c0b8870 2024-01-16T14:55:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0a0900 con 0x7f706c0d0ee0 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f7071e59700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c0a0be0 con 0x7f706c15e210 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f706b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f706b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50162/0 (socket says 172.21.15.26:50162) 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.276+0000 7f706b7fe700 1 -- 172.21.15.26:0/3197462388 learned_addr learned my addr 172.21.15.26:0/3197462388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706affd700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 0x7f706c09f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706bfff700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 0x7f706c09fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 -- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 msgr2=0x7f706c09fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 0x7f706c09fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 -- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 msgr2=0x7f706c09f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 0x7f706c09f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 -- 172.21.15.26:0/3197462388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064009a70 con 0x7f706c0b8870 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706affd700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 0x7f706c09f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706bfff700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 0x7f706c09fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f706b7fe700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c420 tx=0x7f705c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c005d70 con 0x7f706c0b8870 2024-01-16T14:55:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c0070d0 con 0x7f706c0b8870 2024-01-16T14:55:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c0a0ec0 con 0x7f706c0b8870 2024-01-16T14:55:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c007440 con 0x7f706c0b8870 2024-01-16T14:55:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.277+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c0a02e0 con 0x7f706c0b8870 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.278+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c0d3c10 con 0x7f706c0b8870 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.279+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f705c0075a0 con 0x7f706c0b8870 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.279+0000 7f7053fff700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 0x7f705806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.279+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705c088ee0 con 0x7f706c0b8870 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.279+0000 7f706affd700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 0x7f705806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.280+0000 7f706affd700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 0x7f705806d810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f706c0c7750 tx=0x7f7060009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.284+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f705c053e00 con 0x7f706c0b8870 2024-01-16T14:55:26.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.492+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f706c0d0cf0 con 0x7f706c0b8870 2024-01-16T14:55:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.492+0000 7f7053fff700 1 -- 172.21.15.26:0/3197462388 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f705c017020 con 0x7f706c0b8870 2024-01-16T14:55:26.476 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 msgr2=0x7f705806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 0x7f705806d810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f706c0c7750 tx=0x7f7060009040 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 msgr2=0x7f706c0ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c420 tx=0x7f705c00c7e0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 shutdown_connections 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f706c15e210 0x7f706c09fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f705806b350 0x7f705806d810 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f706c0b8870 0x7f706c0ca560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 --2- 172.21.15.26:0/3197462388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f706c0d0ee0 0x7f706c09f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 >> 172.21.15.26:0/3197462388 conn(0x7f706c09d180 msgr2=0x7f706c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 shutdown_connections 2024-01-16T14:55:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.495+0000 7f7071e59700 1 -- 172.21.15.26:0/3197462388 wait complete. 2024-01-16T14:55:26.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 Processor -- start 2024-01-16T14:55:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 -- start start 2024-01-16T14:55:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff82c0726a0 0x7ff82c072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c0b3d90 0x7ff82c072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c15f3e0 con 0x7ff82c06bac0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0affd0 con 0x7ff82c0726a0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff833e6b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c073500 con 0x7ff82c0b3d90 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff831c07700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff82c0726a0 0x7ff82c072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff832408700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff82c06bac0 0x7ff82c06bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41022/0 (socket says 172.21.15.26:41022) 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff832408700 1 -- 172.21.15.26:0/2795418103 learned_addr learned my addr 172.21.15.26:0/2795418103 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.649+0000 7ff831406700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c0b3d90 0x7ff82c072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff831c07700 1 -- 172.21.15.26:0/2795418103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c0b3d90 msgr2=0x7ff82c072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff831c07700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c0b3d90 0x7ff82c072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff831c07700 1 -- 172.21.15.26:0/2795418103 >> v1:172.21.15.26:6789/0 conn(0x7ff82c06bac0 legacy=0x7ff82c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff831c07700 1 -- 172.21.15.26:0/2795418103 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c15e7d0 con 0x7ff82c0726a0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff830c05700 1 -- 172.21.15.26:0/2795418103 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795621222 0 0) 0x7ff82c15f3e0 con 0x7ff82c06bac0 2024-01-16T14:55:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff831c07700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c072a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff828009840 tx=0x7ff828009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7965dfa838e7a4fa server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff830c05700 1 -- 172.21.15.26:0/2795418103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828011830 con 0x7ff82c0726a0 2024-01-16T14:55:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff830c05700 1 -- 172.21.15.26:0/2795418103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828011990 con 0x7ff82c0726a0 2024-01-16T14:55:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff830c05700 1 -- 172.21.15.26:0/2795418103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828011d20 con 0x7ff82c0726a0 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff833e6b700 1 -- 172.21.15.26:0/2795418103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 msgr2=0x7ff82c072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.650+0000 7ff833e6b700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c072a80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff828009840 tx=0x7ff828009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 -- 172.21.15.26:0/2795418103 shutdown_connections 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c072a80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 --2- 172.21.15.26:0/2795418103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c0b3d90 0x7ff82c072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 -- 172.21.15.26:0/2795418103 >> 172.21.15.26:0/2795418103 conn(0x7ff82c0717f0 msgr2=0x7ff82c0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 -- 172.21.15.26:0/2795418103 shutdown_connections 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.651+0000 7ff833e6b700 1 -- 172.21.15.26:0/2795418103 wait complete. 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.652+0000 7ff833e6b700 1 Processor -- start 2024-01-16T14:55:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.652+0000 7ff833e6b700 1 -- start start 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 0x7ff82c1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff82c0b3d90 0x7ff82c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c155120 con 0x7ff82c0726a0 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15ad60 con 0x7ff82c0b3d90 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff833e6b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c155380 con 0x7ff82c06bac0 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50184/0 (socket says 172.21.15.26:50184) 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 -- 172.21.15.26:0/1832902319 learned_addr learned my addr 172.21.15.26:0/1832902319 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831c07700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 0x7ff82c1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff832408700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff82c0b3d90 0x7ff82c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 -- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 msgr2=0x7ff82c1f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 0x7ff82c1f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 -- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff82c0b3d90 msgr2=0x7ff82c15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff82c0b3d90 0x7ff82c15a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 -- 172.21.15.26:0/1832902319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280094a0 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831c07700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 0x7ff82c1f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.653+0000 7ff831406700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e810 tx=0x7ff81c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.654+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c013610 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.654+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c013770 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.654+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c013b20 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.654+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c155660 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.654+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c155c20 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.655+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c075480 con 0x7ff82c0726a0 2024-01-16T14:55:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.655+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff81c013c80 con 0x7ff82c0726a0 2024-01-16T14:55:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.656+0000 7ff8227fc700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 0x7ff81006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.656+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff81c0898e0 con 0x7ff82c0726a0 2024-01-16T14:55:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.656+0000 7ff831c07700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 0x7ff81006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.656+0000 7ff831c07700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 0x7ff81006d840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1f6c20 tx=0x7ff828008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.661+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff81c054750 con 0x7ff82c0726a0 2024-01-16T14:55:26.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.874+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff82c0d09b0 con 0x7ff82c0726a0 2024-01-16T14:55:26.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.874+0000 7ff8227fc700 1 -- 172.21.15.26:0/1832902319 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff81c017020 con 0x7ff82c0726a0 2024-01-16T14:55:26.857 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 msgr2=0x7ff81006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 0x7ff81006d840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1f6c20 tx=0x7ff828008030 comp rx=0 tx=0).stop 2024-01-16T14:55:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 msgr2=0x7ff82c154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e810 tx=0x7ff81c00ebd0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 shutdown_connections 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff82c0b3d90 0x7ff82c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff82c06bac0 0x7ff82c1f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff81006b380 0x7ff81006d840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 --2- 172.21.15.26:0/1832902319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff82c0726a0 0x7ff82c154be0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 >> 172.21.15.26:0/1832902319 conn(0x7ff82c0717f0 msgr2=0x7ff82c06d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 shutdown_connections 2024-01-16T14:55:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:26.876+0000 7ff833e6b700 1 -- 172.21.15.26:0/1832902319 wait complete. 2024-01-16T14:55:26.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.029+0000 7f5c2f529700 1 Processor -- start 2024-01-16T14:55:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.029+0000 7f5c2f529700 1 -- start start 2024-01-16T14:55:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2f529700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c2815a700 0x7f5c280ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2f529700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c280ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2f529700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280c5e10 con 0x7f5c28067140 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2f529700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c2806b710 con 0x7f5c2815a700 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2f529700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2815aae0 con 0x7f5c28066ce0 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2d2c5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c2815a700 0x7f5c280ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2d2c5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c2815a700 0x7f5c280ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50194/0 (socket says 172.21.15.26:50194) 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.030+0000 7f5c2d2c5700 1 -- 172.21.15.26:0/2365657384 learned_addr learned my addr 172.21.15.26:0/2365657384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2cac4700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c280ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2d2c5700 1 -- 172.21.15.26:0/2365657384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 msgr2=0x7f5c280ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2d2c5700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c280ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2d2c5700 1 -- 172.21.15.26:0/2365657384 >> v1:172.21.15.93:6789/0 conn(0x7f5c28067140 legacy=0x7f5c280c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2d2c5700 1 -- 172.21.15.26:0/2365657384 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2806b150 con 0x7f5c2815a700 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c2d2c5700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c2815a700 0x7f5c280ca8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bb90 tx=0x7f5c2400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969946697758bd04 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c1ffff700 1 -- 172.21.15.26:0/2365657384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2400dd50 con 0x7f5c2815a700 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c1ffff700 1 -- 172.21.15.26:0/2365657384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c24010470 con 0x7f5c2815a700 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.031+0000 7f5c1ffff700 1 -- 172.21.15.26:0/2365657384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24010800 con 0x7f5c2815a700 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 -- 172.21.15.26:0/2365657384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c2815a700 msgr2=0x7f5c280ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c2815a700 0x7f5c280ca8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bb90 tx=0x7f5c2400bea0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 -- 172.21.15.26:0/2365657384 shutdown_connections 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c2815a700 0x7f5c280ca8a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 --2- 172.21.15.26:0/2365657384 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c280ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 -- 172.21.15.26:0/2365657384 >> 172.21.15.26:0/2365657384 conn(0x7f5c280b1e80 msgr2=0x7f5c280b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 -- 172.21.15.26:0/2365657384 shutdown_connections 2024-01-16T14:55:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.032+0000 7f5c2f529700 1 -- 172.21.15.26:0/2365657384 wait complete. 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.034+0000 7f5c2f529700 1 Processor -- start 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.034+0000 7f5c2f529700 1 -- start start 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c281458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c2815a700 0x7f5c2815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2815d310 con 0x7f5c28067140 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280c7a20 con 0x7f5c2815a700 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2f529700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280c7d00 con 0x7f5c28066ce0 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50208/0 (socket says 172.21.15.26:50208) 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 -- 172.21.15.26:0/313254752 learned_addr learned my addr 172.21.15.26:0/313254752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2d2c5700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c281458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2dac6700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c2815a700 0x7f5c2815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 -- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 msgr2=0x7f5c281458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c281458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 -- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c2815a700 msgr2=0x7f5c2815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c2815a700 0x7f5c2815cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 -- 172.21.15.26:0/313254752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2400b7f0 con 0x7f5c28067140 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.035+0000 7f5c2cac4700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e810 tx=0x7f5c1800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.036+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1800c070 con 0x7f5c28067140 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.036+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18013610 con 0x7f5c28067140 2024-01-16T14:55:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.036+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c280c7fe0 con 0x7f5c28067140 2024-01-16T14:55:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.036+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c180139c0 con 0x7f5c28067140 2024-01-16T14:55:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.036+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c28153250 con 0x7f5c28067140 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.037+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28075480 con 0x7f5c28067140 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.037+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5c18013b20 con 0x7f5c28067140 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.038+0000 7f5c1dffb700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 0x7f5c0c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.038+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c180898a0 con 0x7f5c28067140 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.038+0000 7f5c2d2c5700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 0x7f5c0c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.039+0000 7f5c2d2c5700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 0x7f5c0c06d810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bee0 tx=0x7f5c24013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.043+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c18054720 con 0x7f5c28067140 2024-01-16T14:55:27.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.249+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c280ca130 con 0x7f5c28067140 2024-01-16T14:55:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.250+0000 7f5c1dffb700 1 -- 172.21.15.26:0/313254752 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c18057d40 con 0x7f5c28067140 2024-01-16T14:55:27.233 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 msgr2=0x7f5c0c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 0x7f5c0c06d810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400bee0 tx=0x7f5c24013000 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 msgr2=0x7f5c280c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800e810 tx=0x7f5c1800eb20 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 shutdown_connections 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c2815a700 0x7f5c2815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c28066ce0 0x7f5c281458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5c0c06b350 0x7f5c0c06d810 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 --2- 172.21.15.26:0/313254752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c28067140 0x7f5c280c7320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.252+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 >> 172.21.15.26:0/313254752 conn(0x7f5c280b1e80 msgr2=0x7f5c280b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.253+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 shutdown_connections 2024-01-16T14:55:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.253+0000 7f5c2f529700 1 -- 172.21.15.26:0/313254752 wait complete. 2024-01-16T14:55:27.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T14:55:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.404+0000 7fd929a41700 1 Processor -- start 2024-01-16T14:55:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.404+0000 7fd929a41700 1 -- start start 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd929a41700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd9240d49d0 0x7fd9240d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd929a41700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9240d29d0 0x7fd9240b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd929a41700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9240d4e50 con 0x7fd92415e460 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd929a41700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9240d2e40 con 0x7fd9240d29d0 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd929a41700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9240b3370 con 0x7fd9240d49d0 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd9237fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9240d29d0 0x7fd9240b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd9237fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9240d29d0 0x7fd9240b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50218/0 (socket says 172.21.15.26:50218) 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd922ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd92415e460 0x7fd9240cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41030/0 (socket says 172.21.15.26:41030) 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd9237fe700 1 -- 172.21.15.26:0/2104486477 learned_addr learned my addr 172.21.15.26:0/2104486477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.405+0000 7fd9227fc700 1 --2- 172.21.15.26:0/2104486477 >> v2:172.21.15.93:3300/0 conn(0x7fd9240d49d0 0x7fd9240d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd9237fe700 1 -- 172.21.15.26:0/2104486477 >> v2:172.21.15.93:3300/0 conn(0x7fd9240d49d0 msgr2=0x7fd9240d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd9237fe700 1 --2- 172.21.15.26:0/2104486477 >> v2:172.21.15.93:3300/0 conn(0x7fd9240d49d0 0x7fd9240d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd9237fe700 1 -- 172.21.15.26:0/2104486477 >> v1:172.21.15.26:6789/0 conn(0x7fd92415e460 legacy=0x7fd9240cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd9237fe700 1 -- 172.21.15.26:0/2104486477 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240cfc40 con 0x7fd9240d29d0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd921ffb700 1 -- 172.21.15.26:0/2104486477 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484621568 0 0) 0x7fd9240d4e50 con 0x7fd92415e460 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd9237fe700 1 --2- 172.21.15.26:0/2104486477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9240d29d0 0x7fd9240b40a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd92408e2f0 tx=0x7fd91c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d06026fb716ee7 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd921ffb700 1 -- 172.21.15.26:0/2104486477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00e070 con 0x7fd9240d29d0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd921ffb700 1 -- 172.21.15.26:0/2104486477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c00b910 con 0x7fd9240d29d0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.406+0000 7fd921ffb700 1 -- 172.21.15.26:0/2104486477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00bcc0 con 0x7fd9240d29d0 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 -- 172.21.15.26:0/2104486477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9240d29d0 msgr2=0x7fd9240b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 --2- 172.21.15.26:0/2104486477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9240d29d0 0x7fd9240b40a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd92408e2f0 tx=0x7fd91c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 -- 172.21.15.26:0/2104486477 shutdown_connections 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 --2- 172.21.15.26:0/2104486477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9240d29d0 0x7fd9240b40a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 --2- 172.21.15.26:0/2104486477 >> v2:172.21.15.93:3300/0 conn(0x7fd9240d49d0 0x7fd9240d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 -- 172.21.15.26:0/2104486477 >> 172.21.15.26:0/2104486477 conn(0x7fd9240d3eb0 msgr2=0x7fd9240d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 -- 172.21.15.26:0/2104486477 shutdown_connections 2024-01-16T14:55:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.407+0000 7fd929a41700 1 -- 172.21.15.26:0/2104486477 wait complete. 2024-01-16T14:55:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.408+0000 7fd929a41700 1 Processor -- start 2024-01-16T14:55:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.408+0000 7fd929a41700 1 -- start start 2024-01-16T14:55:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9240d29d0 0x7fd92415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9240d49d0 0x7fd9241611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240c8a40 con 0x7fd92415e460 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92415ba70 con 0x7fd9240d49d0 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd929a41700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240d1270 con 0x7fd9240d29d0 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd9237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd9237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50230/0 (socket says 172.21.15.26:50230) 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd9237fe700 1 -- 172.21.15.26:0/2142039222 learned_addr learned my addr 172.21.15.26:0/2142039222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd9227fc700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9240d49d0 0x7fd9241611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.409+0000 7fd922ffd700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9240d29d0 0x7fd92415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 -- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9240d29d0 msgr2=0x7fd92415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9240d29d0 0x7fd92415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 -- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9240d49d0 msgr2=0x7fd9241611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9240d49d0 0x7fd9241611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 -- 172.21.15.26:0/2142039222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c009b30 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd9237fe700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b680 tx=0x7fd91c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0048b0 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c0092e0 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9240d1550 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c01c660 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.410+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9240c86d0 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.411+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240c8890 con 0x7fd92415e460 2024-01-16T14:55:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.412+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd91c01c7c0 con 0x7fd92415e460 2024-01-16T14:55:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.412+0000 7fd90bfff700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 0x7fd91006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.412+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd91c08b390 con 0x7fd92415e460 2024-01-16T14:55:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.412+0000 7fd922ffd700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 0x7fd91006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.413+0000 7fd922ffd700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 0x7fd91006d810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd914000f60 tx=0x7fd914007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.417+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd91c056200 con 0x7fd92415e460 2024-01-16T14:55:27.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.646+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd924075480 con 0x7fd92415e460 2024-01-16T14:55:27.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.646+0000 7fd90bfff700 1 -- 172.21.15.26:0/2142039222 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd91c059820 con 0x7fd92415e460 2024-01-16T14:55:27.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.648+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 msgr2=0x7fd91006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.648+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 0x7fd91006d810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd914000f60 tx=0x7fd914007e70 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.648+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 msgr2=0x7fd92415eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.648+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b680 tx=0x7fd91c004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 shutdown_connections 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd91006b350 0x7fd91006d810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92415e460 0x7fd92415eea0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9240d29d0 0x7fd92415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 --2- 172.21.15.26:0/2142039222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9240d49d0 0x7fd9241611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 >> 172.21.15.26:0/2142039222 conn(0x7fd9240d3eb0 msgr2=0x7fd92406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 shutdown_connections 2024-01-16T14:55:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.649+0000 7fd929a41700 1 -- 172.21.15.26:0/2142039222 wait complete. 2024-01-16T14:55:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-16T14:55:27.648 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-01-16T14:55:27.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:27.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-01-16T14:55:27.693 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2024-01-16T14:55:27.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:27.694 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-16T14:55:27.714 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-16T14:55:27.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:27.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.873+0000 7f31abcaf700 1 Processor -- start 2024-01-16T14:55:27.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 -- start start 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 0x7f31a40b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31a40b4d50 0x7f31a4071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a406e550 con 0x7f31a40722d0 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a40b51d0 con 0x7f31a40b4d50 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.874+0000 7f31abcaf700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a406f3b0 con 0x7f31a415abb0 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31a40b4d50 0x7f31a4071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31a40b4d50 0x7f31a4071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50244/0 (socket says 172.21.15.26:50244) 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 -- 172.21.15.26:0/55670007 learned_addr learned my addr 172.21.15.26:0/55670007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a9a4b700 1 --2- 172.21.15.26:0/55670007 >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 0x7f31a40b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 -- 172.21.15.26:0/55670007 >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 msgr2=0x7f31a40b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 --2- 172.21.15.26:0/55670007 >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 0x7f31a40b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 -- 172.21.15.26:0/55670007 >> v1:172.21.15.93:6789/0 conn(0x7f31a40722d0 legacy=0x7f31a415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 -- 172.21.15.26:0/55670007 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4072710 con 0x7f31a40b4d50 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a9a4b700 1 --2- 172.21.15.26:0/55670007 >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 0x7f31a40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a924a700 1 --2- 172.21.15.26:0/55670007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40b4d50 0x7f31a4071d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31a4074990 tx=0x7f3194009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a51e9b306448364 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a8a49700 1 -- 172.21.15.26:0/55670007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194004230 con 0x7f31a40b4d50 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a8a49700 1 -- 172.21.15.26:0/55670007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194004390 con 0x7f31a40b4d50 2024-01-16T14:55:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.875+0000 7f31a8a49700 1 -- 172.21.15.26:0/55670007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940036a0 con 0x7f31a40b4d50 2024-01-16T14:55:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 -- 172.21.15.26:0/55670007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40b4d50 msgr2=0x7f31a4071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 --2- 172.21.15.26:0/55670007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40b4d50 0x7f31a4071d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f31a4074990 tx=0x7f3194009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 -- 172.21.15.26:0/55670007 shutdown_connections 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 --2- 172.21.15.26:0/55670007 >> v2:172.21.15.93:3300/0 conn(0x7f31a415abb0 0x7f31a40b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 --2- 172.21.15.26:0/55670007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40b4d50 0x7f31a4071d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 -- 172.21.15.26:0/55670007 >> 172.21.15.26:0/55670007 conn(0x7f31a40d0aa0 msgr2=0x7f31a40d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 -- 172.21.15.26:0/55670007 shutdown_connections 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.876+0000 7f31abcaf700 1 -- 172.21.15.26:0/55670007 wait complete. 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.877+0000 7f31abcaf700 1 Processor -- start 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.877+0000 7f31abcaf700 1 -- start start 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31a40b4d50 0x7f31a40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31a415abb0 0x7f31a40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a41f4c50 con 0x7f31a40722d0 2024-01-16T14:55:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a41f3e90 con 0x7f31a415abb0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31abcaf700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a41f4170 con 0x7f31a40b4d50 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31a9a4b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31a9a4b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50254/0 (socket says 172.21.15.26:50254) 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.878+0000 7f31a9a4b700 1 -- 172.21.15.26:0/1761429304 learned_addr learned my addr 172.21.15.26:0/1761429304 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31aa24c700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31a415abb0 0x7f31a40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a924a700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31a40b4d50 0x7f31a40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 -- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31a40b4d50 msgr2=0x7f31a40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31a40b4d50 0x7f31a40af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 -- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31a415abb0 msgr2=0x7f31a40afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31a415abb0 0x7f31a40afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 -- 172.21.15.26:0/1761429304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194009550 con 0x7f31a40722d0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31a9a4b700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c7f0 tx=0x7f31a000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a0011800 con 0x7f31a40722d0 2024-01-16T14:55:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a0011960 con 0x7f31a40722d0 2024-01-16T14:55:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a0011d10 con 0x7f31a40722d0 2024-01-16T14:55:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a41f4450 con 0x7f31a40722d0 2024-01-16T14:55:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.879+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a4157380 con 0x7f31a40722d0 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.880+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a40b5c00 con 0x7f31a40722d0 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.881+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f31a0018360 con 0x7f31a40722d0 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.881+0000 7f319a7fc700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 0x7f318806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.881+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31a0089b50 con 0x7f31a40722d0 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.881+0000 7f31a924a700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 0x7f318806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.882+0000 7f31a924a700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 0x7f318806d810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3194009130 tx=0x7f3194008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:27.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:27.886+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31a00549c0 con 0x7f31a40722d0 2024-01-16T14:55:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.148+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31a4075480 con 0x7f31a40722d0 2024-01-16T14:55:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.149+0000 7f319a7fc700 1 -- 172.21.15.26:0/1761429304 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f31a001f090 con 0x7f31a40722d0 2024-01-16T14:55:28.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:28.132 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 msgr2=0x7f318806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 0x7f318806d810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3194009130 tx=0x7f3194008030 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 msgr2=0x7f31a4159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c7f0 tx=0x7f31a000cbb0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 shutdown_connections 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31a415abb0 0x7f31a40afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f318806b350 0x7f318806d810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31a40722d0 0x7f31a4159070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 --2- 172.21.15.26:0/1761429304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31a40b4d50 0x7f31a40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 >> 172.21.15.26:0/1761429304 conn(0x7f31a40d0aa0 msgr2=0x7f31a406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 shutdown_connections 2024-01-16T14:55:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.152+0000 7f31abcaf700 1 -- 172.21.15.26:0/1761429304 wait complete. 2024-01-16T14:55:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:55:28.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.302+0000 7ff0062a9700 1 Processor -- start 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.303+0000 7ff0062a9700 1 -- start start 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7ff0062a9700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff0001448b0 0x7ff00006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7ff0062a9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff000144c90 con 0x7ff0000b44c0 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7ff0062a9700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0000b4c80 con 0x7ff00006f990 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7ff0062a9700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000b4f60 con 0x7ff0001448b0 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7fefff7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0000b44c0 0x7ff00006f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41032/0 (socket says 172.21.15.26:41032) 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7fefff7fe700 1 -- 172.21.15.26:0/1191594643 learned_addr learned my addr 172.21.15.26:0/1191594643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffffff700 1 --2- 172.21.15.26:0/1191594643 >> v2:172.21.15.93:3300/0 conn(0x7ff0001448b0 0x7ff00006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080148277 0 0) 0x7ff000144c90 con 0x7ff0000b44c0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe80035b0 con 0x7ff0000b44c0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffffff700 1 -- 172.21.15.26:0/1191594643 >> v1:172.21.15.93:6789/0 conn(0x7ff00006f990 legacy=0x7ff0000ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923788884 0 0) 0x7ff0000b4c80 con 0x7ff00006f990 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffffff700 1 -- 172.21.15.26:0/1191594643 >> v1:172.21.15.26:6789/0 conn(0x7ff0000b44c0 legacy=0x7ff00006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.304+0000 7feffffff700 1 -- 172.21.15.26:0/1191594643 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000b5240 con 0x7ff0001448b0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7feffffff700 1 --2- 172.21.15.26:0/1191594643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0001448b0 0x7ff00006ec70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feff0009840 tx=0x7feff0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ce43d85857d197 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0011830 con 0x7ff0001448b0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0011990 con 0x7ff0001448b0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7feffeffd700 1 -- 172.21.15.26:0/1191594643 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0011d20 con 0x7ff0001448b0 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 -- 172.21.15.26:0/1191594643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0001448b0 msgr2=0x7ff00006ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 --2- 172.21.15.26:0/1191594643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0001448b0 0x7ff00006ec70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feff0009840 tx=0x7feff0009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 -- 172.21.15.26:0/1191594643 shutdown_connections 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 --2- 172.21.15.26:0/1191594643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0001448b0 0x7ff00006ec70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 -- 172.21.15.26:0/1191594643 >> 172.21.15.26:0/1191594643 conn(0x7ff0000d0bc0 msgr2=0x7ff0000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:28.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.305+0000 7ff0062a9700 1 -- 172.21.15.26:0/1191594643 shutdown_connections 2024-01-16T14:55:28.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.306+0000 7ff0062a9700 1 -- 172.21.15.26:0/1191594643 wait complete. 2024-01-16T14:55:28.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.307+0000 7ff0062a9700 1 Processor -- start 2024-01-16T14:55:28.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.307+0000 7ff0062a9700 1 -- start start 2024-01-16T14:55:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0000b44c0 0x7ff00013bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0001448b0 0x7ff000140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000cb5a0 con 0x7ff00006f990 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00013b150 con 0x7ff0000b44c0 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff0062a9700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00013a5d0 con 0x7ff0001448b0 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50264/0 (socket says 172.21.15.26:50264) 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 -- 172.21.15.26:0/3422739697 learned_addr learned my addr 172.21.15.26:0/3422739697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7fefff7fe700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0000b44c0 0x7ff00013bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7ff004846700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0001448b0 0x7ff000140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 -- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0001448b0 msgr2=0x7ff000140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0001448b0 0x7ff000140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 -- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0000b44c0 msgr2=0x7ff00013bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0000b44c0 0x7ff00013bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 -- 172.21.15.26:0/3422739697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00094a0 con 0x7ff00006f990 2024-01-16T14:55:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.308+0000 7feffffff700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff00090b0 tx=0x7feff0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.309+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff001a7e0 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.309+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff001a940 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.309+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff001acf0 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.309+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00013a8b0 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.309+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00013ae70 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.310+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0000cb6e0 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.310+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7feff0022070 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.311+0000 7feffcff9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 0x7fefec06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.311+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feff008a810 con 0x7ff00006f990 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.311+0000 7fefff7fe700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 0x7fefec06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.311+0000 7fefff7fe700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 0x7fefec06d810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff00006e6c0 tx=0x7feff4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.316+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feff0055710 con 0x7ff00006f990 2024-01-16T14:55:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.528+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff000075480 con 0x7ff00006f990 2024-01-16T14:55:28.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.529+0000 7feffcff9700 1 -- 172.21.15.26:0/3422739697 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7feff0058d30 con 0x7ff00006f990 2024-01-16T14:55:28.512 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:28.512 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:28.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 msgr2=0x7fefec06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 0x7fefec06d810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff00006e6c0 tx=0x7feff4009040 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 msgr2=0x7ff000159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7feff00090b0 tx=0x7feff0004750 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 shutdown_connections 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0001448b0 0x7ff000140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fefec06b350 0x7fefec06d810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff00006f990 0x7ff000159630 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 --2- 172.21.15.26:0/3422739697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0000b44c0 0x7ff00013bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 >> 172.21.15.26:0/3422739697 conn(0x7ff0000d0bc0 msgr2=0x7ff0000cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.531+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 shutdown_connections 2024-01-16T14:55:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.532+0000 7ff0062a9700 1 -- 172.21.15.26:0/3422739697 wait complete. 2024-01-16T14:55:28.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-16T14:54:43.518282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-16T14:54:44.259837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:28.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.686+0000 7f5f1367b700 1 Processor -- start 2024-01-16T14:55:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.687+0000 7f5f1367b700 1 -- start start 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.687+0000 7f5f1367b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.687+0000 7f5f1367b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c15e4a0 con 0x7f5f0c15e0c0 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.687+0000 7f5f1367b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c161bc0 con 0x7f5f0c06b6e0 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.687+0000 7f5f1367b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c0af5d0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11417700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f0c15e0c0 0x7f5f0c06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41048/0 (socket says 172.21.15.26:41048) 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50274/0 (socket says 172.21.15.26:50274) 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11417700 1 -- 172.21.15.26:0/504650313 learned_addr learned my addr 172.21.15.26:0/504650313 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 -- 172.21.15.26:0/504650313 >> v1:172.21.15.93:6789/0 conn(0x7f5f0c06b6e0 legacy=0x7f5f0c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 -- 172.21.15.26:0/504650313 >> v1:172.21.15.26:6789/0 conn(0x7f5f0c15e0c0 legacy=0x7f5f0c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 -- 172.21.15.26:0/504650313 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c162180 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5f11c18700 1 --2- 172.21.15.26:0/504650313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d6a1dbdadeb49f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5efffff700 1 -- 172.21.15.26:0/504650313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400e070 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5efffff700 1 -- 172.21.15.26:0/504650313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0400bbd0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.688+0000 7f5efffff700 1 -- 172.21.15.26:0/504650313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04012430 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 -- 172.21.15.26:0/504650313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 msgr2=0x7f5f0c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 --2- 172.21.15.26:0/504650313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009d90 tx=0x7f5f0400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 -- 172.21.15.26:0/504650313 shutdown_connections 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 --2- 172.21.15.26:0/504650313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c06ffe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 -- 172.21.15.26:0/504650313 >> 172.21.15.26:0/504650313 conn(0x7f5f0c0b27b0 msgr2=0x7f5f0c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 -- 172.21.15.26:0/504650313 shutdown_connections 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.689+0000 7f5f1367b700 1 -- 172.21.15.26:0/504650313 wait complete. 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.690+0000 7f5f1367b700 1 Processor -- start 2024-01-16T14:55:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.690+0000 7f5f1367b700 1 -- start start 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f0c06b6e0 0x7f5f0c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c15e0c0 0x7f5f0c0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c14fab0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c14fd90 con 0x7f5f0c06b6e0 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f1367b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15e6e0 con 0x7f5f0c15e0c0 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f10c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f10c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50286/0 (socket says 172.21.15.26:50286) 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.691+0000 7f5f10c16700 1 -- 172.21.15.26:0/108795899 learned_addr learned my addr 172.21.15.26:0/108795899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f11c18700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c15e0c0 0x7f5f0c0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f11417700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f0c06b6e0 0x7f5f0c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 -- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c15e0c0 msgr2=0x7f5f0c0b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c15e0c0 0x7f5f0c0b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 -- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f0c06b6e0 msgr2=0x7f5f0c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f0c06b6e0 0x7f5f0c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 -- 172.21.15.26:0/108795899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f04009a70 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f10c16700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c700 tx=0x7f5f0000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000052d0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00005430 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c15e9c0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00003880 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.692+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c14f6f0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.693+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c075480 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.694+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5f000039e0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.694+0000 7f5efdffb700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 0x7f5ef006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.694+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f00088fd0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.694+0000 7f5f11417700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 0x7f5ef006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.695+0000 7f5f11417700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 0x7f5ef006d810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f08005910 tx=0x7f5f08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:28.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.699+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f00053f00 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.908+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f0c0b3cf0 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.909+0000 7f5efdffb700 1 -- 172.21.15.26:0/108795899 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f00057520 con 0x7f5f0c0b3f10 2024-01-16T14:55:28.891 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 msgr2=0x7f5ef006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 0x7f5ef006d810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f08005910 tx=0x7f5f08009040 comp rx=0 tx=0).stop 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 msgr2=0x7f5f0c14f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c700 tx=0x7f5f0000cac0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 shutdown_connections 2024-01-16T14:55:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c15e0c0 0x7f5f0c0b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f0c06b6e0 0x7f5f0c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ef006b350 0x7f5ef006d810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 --2- 172.21.15.26:0/108795899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0c0b3f10 0x7f5f0c14f1b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 >> 172.21.15.26:0/108795899 conn(0x7f5f0c0b27b0 msgr2=0x7f5f0c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 shutdown_connections 2024-01-16T14:55:28.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:28.911+0000 7f5f1367b700 1 -- 172.21.15.26:0/108795899 wait complete. 2024-01-16T14:55:28.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-16T14:55:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.065+0000 7f05ee2a6700 1 Processor -- start 2024-01-16T14:55:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.065+0000 7f05ee2a6700 1 -- start start 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05ee2a6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e80b57b0 0x7f05e806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05ee2a6700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 0x7f05e806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05ee2a6700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e806dbe0 con 0x7f05e81478c0 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05ee2a6700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e806b9a0 con 0x7f05e80b57b0 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05ee2a6700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e8147ca0 con 0x7f05e806b510 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.066+0000 7f05e77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e80b57b0 0x7f05e806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05e80b57b0 0x7f05e806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50298/0 (socket says 172.21.15.26:50298) 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 -- 172.21.15.26:0/2565009942 learned_addr learned my addr 172.21.15.26:0/2565009942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05ec843700 1 --2- 172.21.15.26:0/2565009942 >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 0x7f05e806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 -- 172.21.15.26:0/2565009942 >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 msgr2=0x7f05e806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 --2- 172.21.15.26:0/2565009942 >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 0x7f05e806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 -- 172.21.15.26:0/2565009942 >> v1:172.21.15.93:6789/0 conn(0x7f05e81478c0 legacy=0x7f05e8161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 -- 172.21.15.26:0/2565009942 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e81622e0 con 0x7f05e80b57b0 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05ec843700 1 --2- 172.21.15.26:0/2565009942 >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 0x7f05e806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e77fe700 1 --2- 172.21.15.26:0/2565009942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e80b57b0 0x7f05e806afd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f05e806c040 tx=0x7f05dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7136a5e630b262db server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e6ffd700 1 -- 172.21.15.26:0/2565009942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004230 con 0x7f05e80b57b0 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e6ffd700 1 -- 172.21.15.26:0/2565009942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc004390 con 0x7f05e80b57b0 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.067+0000 7f05e6ffd700 1 -- 172.21.15.26:0/2565009942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0036a0 con 0x7f05e80b57b0 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 -- 172.21.15.26:0/2565009942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e80b57b0 msgr2=0x7f05e806afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/2565009942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e80b57b0 0x7f05e806afd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f05e806c040 tx=0x7f05dc009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 -- 172.21.15.26:0/2565009942 shutdown_connections 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/2565009942 >> v2:172.21.15.93:3300/0 conn(0x7f05e806b510 0x7f05e806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/2565009942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e80b57b0 0x7f05e806afd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 -- 172.21.15.26:0/2565009942 >> 172.21.15.26:0/2565009942 conn(0x7f05e80d0aa0 msgr2=0x7f05e80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 -- 172.21.15.26:0/2565009942 shutdown_connections 2024-01-16T14:55:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.068+0000 7f05ee2a6700 1 -- 172.21.15.26:0/2565009942 wait complete. 2024-01-16T14:55:29.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.069+0000 7f05ee2a6700 1 Processor -- start 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 -- start start 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05e80b57b0 0x7f05e81523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05e81478c0 0x7f05e80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8153390 con 0x7f05e806b510 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e813fa80 con 0x7f05e81478c0 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05ee2a6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8150870 con 0x7f05e80b57b0 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05e7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50314/0 (socket says 172.21.15.26:50314) 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.070+0000 7f05e7fff700 1 -- 172.21.15.26:0/744564691 learned_addr learned my addr 172.21.15.26:0/744564691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05ec843700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05e81478c0 0x7f05e80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e77fe700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05e80b57b0 0x7f05e81523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 -- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05e80b57b0 msgr2=0x7f05e81523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05e80b57b0 0x7f05e81523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 -- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05e81478c0 msgr2=0x7f05e80ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05e81478c0 0x7f05e80ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 -- 172.21.15.26:0/744564691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc009550 con 0x7f05e806b510 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e7fff700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f05d800b4d0 tx=0x7f05d800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800bea0 con 0x7f05e806b510 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8011830 con 0x7f05e806b510 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e8152dd0 con 0x7f05e806b510 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8011be0 con 0x7f05e806b510 2024-01-16T14:55:29.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.071+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e8150500 con 0x7f05e806b510 2024-01-16T14:55:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.072+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e8150360 con 0x7f05e806b510 2024-01-16T14:55:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.073+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f05d8011d40 con 0x7f05e806b510 2024-01-16T14:55:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.073+0000 7f05e4ff9700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 0x7f05d406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.074+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05d8089780 con 0x7f05e806b510 2024-01-16T14:55:29.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.074+0000 7f05e77fe700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 0x7f05d406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.074+0000 7f05e77fe700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 0x7f05d406d810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009130 tx=0x7f05dc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.079+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05e8150360 con 0x7f05e806b510 2024-01-16T14:55:29.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.291+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05e8150360 con 0x7f05e806b510 2024-01-16T14:55:29.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.291+0000 7f05e4ff9700 1 -- 172.21.15.26:0/744564691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05e8150360 con 0x7f05e806b510 2024-01-16T14:55:29.274 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 msgr2=0x7f05d406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 0x7f05d406d810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009130 tx=0x7f05dc008030 comp rx=0 tx=0).stop 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 msgr2=0x7f05e813f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f05d800b4d0 tx=0x7f05d800b7e0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 shutdown_connections 2024-01-16T14:55:29.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05e81478c0 0x7f05e80ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f05d406b350 0x7f05d406d810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05e806b510 0x7f05e813f540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 --2- 172.21.15.26:0/744564691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05e80b57b0 0x7f05e81523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 >> 172.21.15.26:0/744564691 conn(0x7f05e80d0aa0 msgr2=0x7f05e81439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 shutdown_connections 2024-01-16T14:55:29.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.294+0000 7f05ee2a6700 1 -- 172.21.15.26:0/744564691 wait complete. 2024-01-16T14:55:29.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-16T14:55:29.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.447+0000 7fd2186d8700 1 Processor -- start 2024-01-16T14:55:29.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.447+0000 7fd2186d8700 1 -- start start 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd2186d8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210144c10 0x7fd21006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd2186d8700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd210145650 0x7fd21006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd2186d8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21006eb80 con 0x7fd210145100 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd2186d8700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21006f500 con 0x7fd210145650 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd2186d8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cd610 con 0x7fd210144c10 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd216c75700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd210145650 0x7fd21006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd215c73700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd210145100 0x7fd210071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41054/0 (socket says 172.21.15.26:41054) 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd215c73700 1 -- 172.21.15.26:0/3333746160 learned_addr learned my addr 172.21.15.26:0/3333746160 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.448+0000 7fd216474700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210144c10 0x7fd21006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd216c75700 1 -- 172.21.15.26:0/3333746160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210144c10 msgr2=0x7fd21006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd216c75700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210144c10 0x7fd21006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd216c75700 1 -- 172.21.15.26:0/3333746160 >> v1:172.21.15.26:6789/0 conn(0x7fd210145100 legacy=0x7fd210071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd216c75700 1 -- 172.21.15.26:0/3333746160 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210160720 con 0x7fd210145650 2024-01-16T14:55:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd216c75700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd21006b510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3a0d5e1da30dad server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd215472700 1 -- 172.21.15.26:0/3333746160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800e070 con 0x7fd210145650 2024-01-16T14:55:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd215472700 1 -- 172.21.15.26:0/3333746160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20800bb20 con 0x7fd210145650 2024-01-16T14:55:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.449+0000 7fd215472700 1 -- 172.21.15.26:0/3333746160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208012430 con 0x7fd210145650 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 -- 172.21.15.26:0/3333746160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 msgr2=0x7fd21006b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd21006b510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd208009d90 tx=0x7fd20800b330 comp rx=0 tx=0).stop 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 -- 172.21.15.26:0/3333746160 shutdown_connections 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd21006b510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 --2- 172.21.15.26:0/3333746160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210144c10 0x7fd21006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 -- 172.21.15.26:0/3333746160 >> 172.21.15.26:0/3333746160 conn(0x7fd2100b2740 msgr2=0x7fd2100b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 -- 172.21.15.26:0/3333746160 shutdown_connections 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.450+0000 7fd2186d8700 1 -- 172.21.15.26:0/3333746160 wait complete. 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.451+0000 7fd2186d8700 1 Processor -- start 2024-01-16T14:55:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.451+0000 7fd2186d8700 1 -- start start 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210144c10 0x7fd21013f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210145100 0x7fd210143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210142d70 con 0x7fd210145650 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210142160 con 0x7fd210144c10 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd2186d8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd210142440 con 0x7fd210145100 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50322/0 (socket says 172.21.15.26:50322) 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 -- 172.21.15.26:0/164151133 learned_addr learned my addr 172.21.15.26:0/164151133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216474700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210144c10 0x7fd21013f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd215c73700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210145100 0x7fd210143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 -- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210145100 msgr2=0x7fd210143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210145100 0x7fd210143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 -- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210144c10 msgr2=0x7fd21013f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210144c10 0x7fd21013f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.452+0000 7fd216c75700 1 -- 172.21.15.26:0/164151133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009a70 con 0x7fd210145650 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd216c75700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd208014040 tx=0x7fd208004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208003ea0 con 0x7fd210145650 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208020070 con 0x7fd210145650 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080276c0 con 0x7fd210145650 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd210142720 con 0x7fd210145650 2024-01-16T14:55:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.453+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210141d80 con 0x7fd210145650 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.454+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2101444a0 con 0x7fd210145650 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.454+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd208012430 con 0x7fd210145650 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.455+0000 7fd206ffd700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 0x7fd1f406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.455+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd208022070 con 0x7fd210145650 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.455+0000 7fd216474700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 0x7fd1f406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.456+0000 7fd216474700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 0x7fd1f406d810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009b70 tx=0x7fd20c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.460+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2080567b0 con 0x7fd210145650 2024-01-16T14:55:29.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.671+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd210075480 con 0x7fd210145650 2024-01-16T14:55:29.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.671+0000 7fd206ffd700 1 -- 172.21.15.26:0/164151133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd20802c0a0 con 0x7fd210145650 2024-01-16T14:55:29.654 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 msgr2=0x7fd1f406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 0x7fd1f406d810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009b70 tx=0x7fd20c008040 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 msgr2=0x7fd2100a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd208014040 tx=0x7fd208004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 shutdown_connections 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd1f406b350 0x7fd1f406d810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd210145650 0x7fd2100a9570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd210144c10 0x7fd21013f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 --2- 172.21.15.26:0/164151133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd210145100 0x7fd210143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 >> 172.21.15.26:0/164151133 conn(0x7fd2100b2740 msgr2=0x7fd2100b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 shutdown_connections 2024-01-16T14:55:29.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.674+0000 7fd2186d8700 1 -- 172.21.15.26:0/164151133 wait complete. 2024-01-16T14:55:29.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.824+0000 7fc3f925b700 1 Processor -- start 2024-01-16T14:55:29.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.824+0000 7fc3f925b700 1 -- start start 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f925b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3f40b57b0 0x7fc3f406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f925b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f406b510 0x7fc3f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f925b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f406b970 con 0x7fc3f41478c0 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f925b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f4147ca0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f925b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f406dbe0 con 0x7fc3f406b510 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3f40b57b0 0x7fc3f406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3f40b57b0 0x7fc3f406afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50332/0 (socket says 172.21.15.26:50332) 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 -- 172.21.15.26:0/1317607927 learned_addr learned my addr 172.21.15.26:0/1317607927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f359e700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f406b510 0x7fc3f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 -- 172.21.15.26:0/1317607927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f406b510 msgr2=0x7fc3f406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f406b510 0x7fc3f406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 -- 172.21.15.26:0/1317607927 >> v1:172.21.15.93:6789/0 conn(0x7fc3f41478c0 legacy=0x7fc3f4161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.825+0000 7fc3f259c700 1 -- 172.21.15.26:0/1317607927 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f41622e0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f259c700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f406afd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098f0 tx=0x7fc3dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5cb206cde7d760 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f1d9b700 1 -- 172.21.15.26:0/1317607927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00b8a0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f1d9b700 1 -- 172.21.15.26:0/1317607927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00ba00 con 0x7fc3f40b57b0 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f1d9b700 1 -- 172.21.15.26:0/1317607927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00bd90 con 0x7fc3f40b57b0 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 -- 172.21.15.26:0/1317607927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 msgr2=0x7fc3f406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f406afd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098f0 tx=0x7fc3dc009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 -- 172.21.15.26:0/1317607927 shutdown_connections 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f406b510 0x7fc3f406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1317607927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f406afd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 -- 172.21.15.26:0/1317607927 >> 172.21.15.26:0/1317607927 conn(0x7fc3f40d0aa0 msgr2=0x7fc3f40d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 -- 172.21.15.26:0/1317607927 shutdown_connections 2024-01-16T14:55:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.826+0000 7fc3f925b700 1 -- 172.21.15.26:0/1317607927 wait complete. 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.828+0000 7fc3f925b700 1 Processor -- start 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.828+0000 7fc3f925b700 1 -- start start 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f406b510 0x7fc3f41609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f41478c0 0x7fc3f4140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41f3ec0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41f35f0 con 0x7fc3f406b510 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f925b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40c6f60 con 0x7fc3f41478c0 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50348/0 (socket says 172.21.15.26:50348) 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 -- 172.21.15.26:0/1447259859 learned_addr learned my addr 172.21.15.26:0/1447259859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f2d9d700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f406b510 0x7fc3f41609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f359e700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f41478c0 0x7fc3f4140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 -- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f41478c0 msgr2=0x7fc3f4140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f41478c0 0x7fc3f4140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 -- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f406b510 msgr2=0x7fc3f41609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f406b510 0x7fc3f41609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.829+0000 7fc3f259c700 1 -- 172.21.15.26:0/1447259859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc009550 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3f259c700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc004480 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc01f070 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f40c7240 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0038c0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.830+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41f3ae0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.831+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f41f4000 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.831+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc3dc021030 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.832+0000 7fc3eb7fe700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 0x7fc3d406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.832+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3dc08b0e0 con 0x7fc3f40b57b0 2024-01-16T14:55:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.832+0000 7fc3f2d9d700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 0x7fc3d406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.832+0000 7fc3f2d9d700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 0x7fc3d406d810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b4dd0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:29.837+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3f41f4000 con 0x7fc3f40b57b0 2024-01-16T14:55:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.095+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f41f4000 con 0x7fc3f40b57b0 2024-01-16T14:55:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.095+0000 7fc3eb7fe700 1 -- 172.21.15.26:0/1447259859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fc3f41f4000 con 0x7fc3f40b57b0 2024-01-16T14:55:30.078 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:30.078 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 msgr2=0x7fc3d406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 0x7fc3d406d810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40b4dd0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 msgr2=0x7fc3f40c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc004750 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 shutdown_connections 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc3f41478c0 0x7fc3f4140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f406b510 0x7fc3f41609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc3d406b350 0x7fc3d406d810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 --2- 172.21.15.26:0/1447259859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3f40b57b0 0x7fc3f40c6670 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 >> 172.21.15.26:0/1447259859 conn(0x7fc3f40d0aa0 msgr2=0x7fc3f41451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 shutdown_connections 2024-01-16T14:55:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.098+0000 7fc3f925b700 1 -- 172.21.15.26:0/1447259859 wait complete. 2024-01-16T14:55:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:55:30.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.248+0000 7f4ca1d9f700 1 Processor -- start 2024-01-16T14:55:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.248+0000 7f4ca1d9f700 1 -- start start 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4ca1d9f700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4c9c15e000 0x7f4c9c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4ca1d9f700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4ca1d9f700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0cfd60 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4ca1d9f700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c0cf4b0 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4ca1d9f700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c06f430 con 0x7f4c9c15e000 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50360/0 (socket says 172.21.15.26:50360) 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 -- 172.21.15.26:0/3295408642 learned_addr learned my addr 172.21.15.26:0/3295408642 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9b7fe700 1 --2- 172.21.15.26:0/3295408642 >> v2:172.21.15.93:3300/0 conn(0x7f4c9c15e000 0x7f4c9c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 -- 172.21.15.26:0/3295408642 >> v2:172.21.15.93:3300/0 conn(0x7f4c9c15e000 msgr2=0x7f4c9c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 --2- 172.21.15.26:0/3295408642 >> v2:172.21.15.93:3300/0 conn(0x7f4c9c15e000 0x7f4c9c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 -- 172.21.15.26:0/3295408642 >> v1:172.21.15.93:6789/0 conn(0x7f4c9c0d0f60 legacy=0x7f4c9c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 -- 172.21.15.26:0/3295408642 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0cf1d0 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9affd700 1 --2- 172.21.15.26:0/3295408642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06bf70 tx=0x7f4c90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58767eceb525f62d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.249+0000 7f4c9a7fc700 1 -- 172.21.15.26:0/3295408642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90011830 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4c9a7fc700 1 -- 172.21.15.26:0/3295408642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90011990 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4c9a7fc700 1 -- 172.21.15.26:0/3295408642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90011d20 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/3295408642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0ae0 msgr2=0x7f4c9c06f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/3295408642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c06bf70 tx=0x7f4c90009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/3295408642 shutdown_connections 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/3295408642 >> v2:172.21.15.93:3300/0 conn(0x7f4c9c15e000 0x7f4c9c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/3295408642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0ae0 0x7f4c9c06f6f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/3295408642 >> 172.21.15.26:0/3295408642 conn(0x7f4c9c0b3720 msgr2=0x7f4c9c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/3295408642 shutdown_connections 2024-01-16T14:55:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.250+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/3295408642 wait complete. 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.251+0000 7f4ca1d9f700 1 Processor -- start 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 -- start start 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c9c0d0ae0 0x7f4c9c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c9c15e000 0x7f4c9c160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1615e0 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c160d40 con 0x7f4c9c15e000 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.252+0000 7f4ca1d9f700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c161020 con 0x7f4c9c0d0ae0 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50364/0 (socket says 172.21.15.26:50364) 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 -- 172.21.15.26:0/887919323 learned_addr learned my addr 172.21.15.26:0/887919323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9b7fe700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c9c0d0ae0 0x7f4c9c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9bfff700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c9c15e000 0x7f4c9c160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 -- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c9c0d0ae0 msgr2=0x7f4c9c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c9c0d0ae0 0x7f4c9c15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 -- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c9c15e000 msgr2=0x7f4c9c160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c9c15e000 0x7f4c9c160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 -- 172.21.15.26:0/887919323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90009550 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c9affd700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009130 tx=0x7f4c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90004510 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9001e070 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c161300 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900038a0 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.253+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c160a60 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.254+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c0d1a00 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.255+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4c90003a00 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.256+0000 7f4c83fff700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 0x7f4c8806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.256+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c9008aa40 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.256+0000 7f4c9b7fe700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 0x7f4c8806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.256+0000 7f4c9b7fe700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 0x7f4c8806d810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0b75f0 tx=0x7f4c8c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.261+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c9008ce60 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.473+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c9c0b5900 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.474+0000 7f4c83fff700 1 -- 172.21.15.26:0/887919323 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f4c9c0b5900 con 0x7f4c9c0d0f60 2024-01-16T14:55:30.457 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:30.458 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 msgr2=0x7f4c8806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 0x7f4c8806d810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0b75f0 tx=0x7f4c8c009210 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 msgr2=0x7f4c9c15ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009130 tx=0x7f4c90004750 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 shutdown_connections 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c9c15e000 0x7f4c9c160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c9c0d0ae0 0x7f4c9c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4c8806b350 0x7f4c8806d810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 --2- 172.21.15.26:0/887919323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c9c0d0f60 0x7f4c9c15ffe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 >> 172.21.15.26:0/887919323 conn(0x7f4c9c0b3720 msgr2=0x7f4c9c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 shutdown_connections 2024-01-16T14:55:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.477+0000 7f4ca1d9f700 1 -- 172.21.15.26:0/887919323 wait complete. 2024-01-16T14:55:30.474 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-16T14:55:30.474 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.628+0000 7efc4ec49700 1 Processor -- start 2024-01-16T14:55:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.628+0000 7efc4ec49700 1 -- start start 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4ec49700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc480b57b0 0x7efc4806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4ec49700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc4806b510 0x7efc4806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4ec49700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4806b970 con 0x7efc481478c0 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4ec49700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc48147ca0 con 0x7efc480b57b0 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4ec49700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4806dbe0 con 0x7efc4806b510 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc47fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc480b57b0 0x7efc4806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4c9e5700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efc481478c0 0x7efc48161a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41066/0 (socket says 172.21.15.26:41066) 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.629+0000 7efc4c9e5700 1 -- 172.21.15.26:0/2298516972 learned_addr learned my addr 172.21.15.26:0/2298516972 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc4d1e6700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc4806b510 0x7efc4806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc47fff700 1 -- 172.21.15.26:0/2298516972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc4806b510 msgr2=0x7efc4806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc47fff700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc4806b510 0x7efc4806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc47fff700 1 -- 172.21.15.26:0/2298516972 >> v1:172.21.15.26:6789/0 conn(0x7efc481478c0 legacy=0x7efc48161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc477fe700 1 -- 172.21.15.26:0/2298516972 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173006496 0 0) 0x7efc4806b970 con 0x7efc481478c0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc47fff700 1 -- 172.21.15.26:0/2298516972 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc481622e0 con 0x7efc480b57b0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc47fff700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc480b57b0 0x7efc4806afd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc30009840 tx=0x7efc30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef42b0444991463 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc477fe700 1 -- 172.21.15.26:0/2298516972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30011830 con 0x7efc480b57b0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc477fe700 1 -- 172.21.15.26:0/2298516972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30011990 con 0x7efc480b57b0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.630+0000 7efc477fe700 1 -- 172.21.15.26:0/2298516972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30011d20 con 0x7efc480b57b0 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 -- 172.21.15.26:0/2298516972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc480b57b0 msgr2=0x7efc4806afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc480b57b0 0x7efc4806afd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc30009840 tx=0x7efc30009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 -- 172.21.15.26:0/2298516972 shutdown_connections 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc4806b510 0x7efc4806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 --2- 172.21.15.26:0/2298516972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc480b57b0 0x7efc4806afd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 -- 172.21.15.26:0/2298516972 >> 172.21.15.26:0/2298516972 conn(0x7efc480d0aa0 msgr2=0x7efc480d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 -- 172.21.15.26:0/2298516972 shutdown_connections 2024-01-16T14:55:30.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.631+0000 7efc4ec49700 1 -- 172.21.15.26:0/2298516972 wait complete. 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.632+0000 7efc4ec49700 1 Processor -- start 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.632+0000 7efc4ec49700 1 -- start start 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc480b57b0 0x7efc480b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc481478c0 0x7efc480c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc481531c0 con 0x7efc4806b510 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc480b1390 con 0x7efc481478c0 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4ec49700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc480b1670 con 0x7efc480b57b0 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4c9e5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4c9e5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50388/0 (socket says 172.21.15.26:50388) 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4c9e5700 1 -- 172.21.15.26:0/234348475 learned_addr learned my addr 172.21.15.26:0/234348475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc47fff700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc480b57b0 0x7efc480b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.633+0000 7efc4d1e6700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc481478c0 0x7efc480c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 -- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc480b57b0 msgr2=0x7efc480b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc480b57b0 0x7efc480b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 -- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc481478c0 msgr2=0x7efc480c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc481478c0 0x7efc480c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 -- 172.21.15.26:0/234348475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300094a0 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4c9e5700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c550 tx=0x7efc3800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38004c80 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc38004e00 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc480b1950 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38007310 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.634+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc481416e0 con 0x7efc4806b510 2024-01-16T14:55:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.635+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc48075480 con 0x7efc4806b510 2024-01-16T14:55:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.636+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7efc38007470 con 0x7efc4806b510 2024-01-16T14:55:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.636+0000 7efc457fa700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 0x7efc2806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.636+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc38089840 con 0x7efc4806b510 2024-01-16T14:55:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.636+0000 7efc47fff700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 0x7efc2806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.637+0000 7efc47fff700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 0x7efc2806d810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc300090b0 tx=0x7efc30008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:30.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.641+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc380546b0 con 0x7efc4806b510 2024-01-16T14:55:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.898+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc480b1ae0 con 0x7efc4806b510 2024-01-16T14:55:30.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.899+0000 7efc457fa700 1 -- 172.21.15.26:0/234348475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7efc38057cd0 con 0x7efc4806b510 2024-01-16T14:55:30.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:30.882 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 msgr2=0x7efc2806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 0x7efc2806d810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc300090b0 tx=0x7efc30008030 comp rx=0 tx=0).stop 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 msgr2=0x7efc481411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c550 tx=0x7efc3800c860 comp rx=0 tx=0).stop 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 shutdown_connections 2024-01-16T14:55:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc481478c0 0x7efc480c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7efc2806b350 0x7efc2806d810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc4806b510 0x7efc481411a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 --2- 172.21.15.26:0/234348475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc480b57b0 0x7efc480b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 >> 172.21.15.26:0/234348475 conn(0x7efc480d0aa0 msgr2=0x7efc481617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 shutdown_connections 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:30.901+0000 7efc4ec49700 1 -- 172.21.15.26:0/234348475 wait complete. 2024-01-16T14:55:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:55:30.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:31.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.051+0000 7f97435a6700 1 Processor -- start 2024-01-16T14:55:31.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.051+0000 7f97435a6700 1 -- start start 2024-01-16T14:55:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f97435a6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 0x7f973c168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f97435a6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c06dbe0 con 0x7f973c1478c0 2024-01-16T14:55:31.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f97435a6700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c1622e0 con 0x7f973c0b57b0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f97435a6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c147ca0 con 0x7f973c06b6e0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741342700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f973c1478c0 0x7f973c161a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41074/0 (socket says 172.21.15.26:41074) 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741342700 1 -- 172.21.15.26:0/1373020681 learned_addr learned my addr 172.21.15.26:0/1373020681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741b43700 1 --2- 172.21.15.26:0/1373020681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 0x7f973c168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981327219 0 0) 0x7f973c06dbe0 con 0x7f973c1478c0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97280035b0 con 0x7f973c1478c0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351575087 0 0) 0x7f973c1622e0 con 0x7f973c0b57b0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f973c06dbe0 con 0x7f973c0b57b0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741b43700 1 -- 172.21.15.26:0/1373020681 >> v1:172.21.15.93:6789/0 conn(0x7f973c0b57b0 legacy=0x7f973c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741b43700 1 -- 172.21.15.26:0/1373020681 >> v1:172.21.15.26:6789/0 conn(0x7f973c1478c0 legacy=0x7f973c161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.053+0000 7f9741b43700 1 -- 172.21.15.26:0/1373020681 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c1625c0 con 0x7f973c06b6e0 2024-01-16T14:55:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f9741b43700 1 --2- 172.21.15.26:0/1373020681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 0x7f973c168e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9734009dc0 tx=0x7f973400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c4b2466616aa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400e070 con 0x7f973c06b6e0 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400bb20 con 0x7f973c06b6e0 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f972ffff700 1 -- 172.21.15.26:0/1373020681 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734012430 con 0x7f973c06b6e0 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f97435a6700 1 -- 172.21.15.26:0/1373020681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 msgr2=0x7f973c168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f97435a6700 1 --2- 172.21.15.26:0/1373020681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 0x7f973c168e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9734009dc0 tx=0x7f973400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f97435a6700 1 -- 172.21.15.26:0/1373020681 shutdown_connections 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f97435a6700 1 --2- 172.21.15.26:0/1373020681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c06b6e0 0x7f973c168e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.054+0000 7f97435a6700 1 -- 172.21.15.26:0/1373020681 >> 172.21.15.26:0/1373020681 conn(0x7f973c0d0aa0 msgr2=0x7f973c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.055+0000 7f97435a6700 1 -- 172.21.15.26:0/1373020681 shutdown_connections 2024-01-16T14:55:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.055+0000 7f97435a6700 1 -- 172.21.15.26:0/1373020681 wait complete. 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.056+0000 7f97435a6700 1 Processor -- start 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.056+0000 7f97435a6700 1 -- start start 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 0x7f973c146850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c1478c0 0x7f973c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c13adb0 con 0x7f973c06b6e0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c146d90 con 0x7f973c0b57b0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c13aad0 con 0x7f973c1478c0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50404/0 (socket says 172.21.15.26:50404) 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 -- 172.21.15.26:0/2999550626 learned_addr learned my addr 172.21.15.26:0/2999550626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741b43700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c1478c0 0x7f973c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9740b41700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 0x7f973c146850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 -- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c1478c0 msgr2=0x7f973c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c1478c0 0x7f973c0aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 -- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 msgr2=0x7f973c146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 0x7f973c146850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 -- 172.21.15.26:0/2999550626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009a70 con 0x7f973c06b6e0 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9740b41700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 0x7f973c146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f9741342700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f973000c700 tx=0x7f973000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730005d70 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300070d0 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.057+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c140980 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.058+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730007460 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.058+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c140f40 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.059+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c1407e0 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.059+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f97300075c0 con 0x7f973c06b6e0 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.060+0000 7f972dffb700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 0x7f972006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.060+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97300890c0 con 0x7f973c06b6e0 2024-01-16T14:55:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.060+0000 7f9740b41700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 0x7f972006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.061+0000 7f9740b41700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 0x7f972006d810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f973c0afa30 tx=0x7f9738009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.064+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f973c1407e0 con 0x7f973c06b6e0 2024-01-16T14:55:31.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.318+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f973c1407e0 con 0x7f973c06b6e0 2024-01-16T14:55:31.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.319+0000 7f972dffb700 1 -- 172.21.15.26:0/2999550626 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f973c1407e0 con 0x7f973c06b6e0 2024-01-16T14:55:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:31.302 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1008727158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1008727158},{"type":"v1","addr":"172.21.15.26:6834","nonce":1008727158}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1393812800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1393812800},{"type":"v1","addr":"172.21.15.26:6835","nonce":1393812800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/3882679114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3882679114},{"type":"v1","addr":"172.21.15.93:6837","nonce":3882679114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:54:45.280907+0000","modified":"2024-01-16T14:54:48.302392+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6836/2658521429","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":2658521429},{"type":"v1","addr":"172.21.15.93:6836","nonce":2658521429}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 msgr2=0x7f972006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 0x7f972006d810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f973c0afa30 tx=0x7f9738009040 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 msgr2=0x7f973c141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f973000c700 tx=0x7f973000ca10 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 shutdown_connections 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f973c1478c0 0x7f973c0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f972006b350 0x7f972006d810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f973c06b6e0 0x7f973c141460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 --2- 172.21.15.26:0/2999550626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f973c0b57b0 0x7f973c146850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 >> 172.21.15.26:0/2999550626 conn(0x7f973c0d0aa0 msgr2=0x7f973c1451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 shutdown_connections 2024-01-16T14:55:31.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.322+0000 7f97435a6700 1 -- 172.21.15.26:0/2999550626 wait complete. 2024-01-16T14:55:31.306 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2024-01-16T14:55:31.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.472+0000 7f8c5573b700 1 Processor -- start 2024-01-16T14:55:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.472+0000 7f8c5573b700 1 -- start start 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.472+0000 7f8c5573b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 0x7f8c500cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.472+0000 7f8c5573b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c5573b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500ce9a0 con 0x7f8c5015e080 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c5573b700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500d14b0 con 0x7f8c500d2b50 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c5573b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500cec40 con 0x7f8c500d0740 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4e7fc700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 0x7f8c500cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:33796/0 (socket says 172.21.15.26:33796) 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 -- 172.21.15.26:0/2140458564 learned_addr learned my addr 172.21.15.26:0/2140458564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:31.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4effd700 1 --1- 172.21.15.26:0/2140458564 >> v1:172.21.15.93:6789/0 conn(0x7f8c5015e080 0x7f8c500d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:51134/0 (socket says 172.21.15.26:51134) 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 -- 172.21.15.26:0/2140458564 >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 msgr2=0x7f8c500cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 --2- 172.21.15.26:0/2140458564 >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 0x7f8c500cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 -- 172.21.15.26:0/2140458564 >> v1:172.21.15.93:6789/0 conn(0x7f8c5015e080 legacy=0x7f8c500d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 -- 172.21.15.26:0/2140458564 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500cef20 con 0x7f8c500d0740 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4e7fc700 1 --2- 172.21.15.26:0/2140458564 >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 0x7f8c500cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4dffb700 1 -- 172.21.15.26:0/2140458564 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652582396 0 0) 0x7f8c500ce9a0 con 0x7f8c5015e080 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.473+0000 7f8c4f7fe700 1 --2- 172.21.15.26:0/2140458564 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009e50 tx=0x7f8c4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae33615325aa5ade server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c4dffb700 1 -- 172.21.15.26:0/2140458564 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800e070 con 0x7f8c500d0740 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c4dffb700 1 -- 172.21.15.26:0/2140458564 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800b910 con 0x7f8c500d0740 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c4dffb700 1 -- 172.21.15.26:0/2140458564 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800bcc0 con 0x7f8c500d0740 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 -- 172.21.15.26:0/2140458564 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 msgr2=0x7f8c500d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 --2- 172.21.15.26:0/2140458564 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009e50 tx=0x7f8c4800b330 comp rx=0 tx=0).stop 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 -- 172.21.15.26:0/2140458564 shutdown_connections 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 --2- 172.21.15.26:0/2140458564 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d0740 0x7f8c500d0bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 --2- 172.21.15.26:0/2140458564 >> v2:172.21.15.93:3300/0 conn(0x7f8c500d2b50 0x7f8c500cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 -- 172.21.15.26:0/2140458564 >> 172.21.15.26:0/2140458564 conn(0x7f8c5009d180 msgr2=0x7f8c500d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 -- 172.21.15.26:0/2140458564 shutdown_connections 2024-01-16T14:55:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.474+0000 7f8c5573b700 1 -- 172.21.15.26:0/2140458564 wait complete. 2024-01-16T14:55:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.476+0000 7f8c5573b700 1 Processor -- start 2024-01-16T14:55:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.476+0000 7f8c5573b700 1 -- start start 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.476+0000 7f8c5573b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.476+0000 7f8c5573b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d2b50 0x7f8c500ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c5573b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c5015e080 0x7f8c500b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c5573b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50149ef0 con 0x7f8c500d0740 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c5573b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500cd580 con 0x7f8c5015e080 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c5573b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500cd860 con 0x7f8c500d2b50 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50418/0 (socket says 172.21.15.26:50418) 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 -- 172.21.15.26:0/1351888606 learned_addr learned my addr 172.21.15.26:0/1351888606 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4e7fc700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d2b50 0x7f8c500ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4f7fe700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c5015e080 0x7f8c500b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 -- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d2b50 msgr2=0x7f8c500ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d2b50 0x7f8c500ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 -- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c5015e080 msgr2=0x7f8c500b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c5015e080 0x7f8c500b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 -- 172.21.15.26:0/1351888606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440094a0 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c4effd700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c550 tx=0x7f8c4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.477+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40004c80 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.478+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40004e00 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.478+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c48009b30 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.478+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40007310 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.478+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50147370 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.479+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50075480 con 0x7f8c500d0740 2024-01-16T14:55:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.479+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8c40007470 con 0x7f8c500d0740 2024-01-16T14:55:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.479+0000 7f8c377fe700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 0x7f8c3c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.480+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c40089820 con 0x7f8c500d0740 2024-01-16T14:55:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.480+0000 7f8c4e7fc700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 0x7f8c3c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.480+0000 7f8c4e7fc700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 0x7f8c3c06d810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c50149600 tx=0x7f8c44009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.484+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c40054600 con 0x7f8c500d0740 2024-01-16T14:55:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.698+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c50161750 con 0x7f8c500d0740 2024-01-16T14:55:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.699+0000 7f8c377fe700 1 -- 172.21.15.26:0/1351888606 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f8c40089020 con 0x7f8c500d0740 2024-01-16T14:55:31.682 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:31.682 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:54:46.849179+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:54:43.518282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-16T14:54:44.259837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 msgr2=0x7f8c3c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 0x7f8c3c06d810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c50149600 tx=0x7f8c44009210 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 msgr2=0x7f8c500ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c550 tx=0x7f8c4000c860 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 shutdown_connections 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c5015e080 0x7f8c500b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f8c3c06b350 0x7f8c3c06d810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c500d0740 0x7f8c500ca1d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 --2- 172.21.15.26:0/1351888606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c500d2b50 0x7f8c500ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 >> 172.21.15.26:0/1351888606 conn(0x7f8c5009d180 msgr2=0x7f8c5015e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 shutdown_connections 2024-01-16T14:55:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.702+0000 7f8c5573b700 1 -- 172.21.15.26:0/1351888606 wait complete. 2024-01-16T14:55:31.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-16T14:55:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.853+0000 7fbdbd83d700 1 Processor -- start 2024-01-16T14:55:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.853+0000 7fbdbd83d700 1 -- start start 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdbd83d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb815e1e0 0x7fbdb80d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdbd83d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdb80d2810 0x7fbdb80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdbd83d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80d1760 con 0x7fbdb80d0e10 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdbd83d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb80d1270 con 0x7fbdb80d2810 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdbd83d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80d1ee0 con 0x7fbdb815e1e0 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdb80d2810 0x7fbdb80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdb80d2810 0x7fbdb80b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50430/0 (socket says 172.21.15.26:50430) 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 -- 172.21.15.26:0/2396706474 learned_addr learned my addr 172.21.15.26:0/2396706474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb6ffd700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb815e1e0 0x7fbdb80d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 -- 172.21.15.26:0/2396706474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb815e1e0 msgr2=0x7fbdb80d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb815e1e0 0x7fbdb80d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 -- 172.21.15.26:0/2396706474 >> v1:172.21.15.93:6789/0 conn(0x7fbdb80d0e10 legacy=0x7fbdb80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.854+0000 7fbdb77fe700 1 -- 172.21.15.26:0/2396706474 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80d1480 con 0x7fbdb80d2810 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdb77fe700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d2810 0x7fbdb80b3ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e50 tx=0x7fbdb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeda7242d3d839f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdb5ffb700 1 -- 172.21.15.26:0/2396706474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000e070 con 0x7fbdb80d2810 2024-01-16T14:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdb5ffb700 1 -- 172.21.15.26:0/2396706474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb0003a20 con 0x7fbdb80d2810 2024-01-16T14:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdb5ffb700 1 -- 172.21.15.26:0/2396706474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0003dd0 con 0x7fbdb80d2810 2024-01-16T14:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdbd83d700 1 -- 172.21.15.26:0/2396706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d2810 msgr2=0x7fbdb80b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.855+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d2810 0x7fbdb80b3ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0009e50 tx=0x7fbdb000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 -- 172.21.15.26:0/2396706474 shutdown_connections 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d2810 0x7fbdb80b3ec0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/2396706474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb815e1e0 0x7fbdb80d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 -- 172.21.15.26:0/2396706474 >> 172.21.15.26:0/2396706474 conn(0x7fbdb80d3dd0 msgr2=0x7fbdb80d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 -- 172.21.15.26:0/2396706474 shutdown_connections 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.856+0000 7fbdbd83d700 1 -- 172.21.15.26:0/2396706474 wait complete. 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.857+0000 7fbdbd83d700 1 Processor -- start 2024-01-16T14:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.857+0000 7fbdbd83d700 1 -- start start 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb80d2810 0x7fbdb80b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdb815e1e0 0x7fbdb81f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b8b80 con 0x7fbdb80d0e10 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb81f4910 con 0x7fbdb815e1e0 2024-01-16T14:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdbd83d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b69c0 con 0x7fbdb80d2810 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50442/0 (socket says 172.21.15.26:50442) 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 -- 172.21.15.26:0/390913935 learned_addr learned my addr 172.21.15.26:0/390913935 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb77fe700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdb815e1e0 0x7fbdb81f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb67fc700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb80d2810 0x7fbdb80b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 -- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb80d2810 msgr2=0x7fbdb80b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb80d2810 0x7fbdb80b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 -- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdb815e1e0 msgr2=0x7fbdb81f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdb815e1e0 0x7fbdb81f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.858+0000 7fbdb6ffd700 1 -- 172.21.15.26:0/390913935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb0009b30 con 0x7fbdb80d0e10 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbdb6ffd700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d800 tx=0x7fbda800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80125f0 con 0x7fbdb80d0e10 2024-01-16T14:55:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8012750 con 0x7fbdb80d0e10 2024-01-16T14:55:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb81f4bd0 con 0x7fbdb80d0e10 2024-01-16T14:55:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8012b00 con 0x7fbdb80d0e10 2024-01-16T14:55:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.859+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb81f5190 con 0x7fbdb80d0e10 2024-01-16T14:55:31.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.860+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb80b6b80 con 0x7fbdb80d0e10 2024-01-16T14:55:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.860+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbda8012c60 con 0x7fbdb80d0e10 2024-01-16T14:55:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.861+0000 7fbd9ffff700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 0x7fbda406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.861+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda8089b40 con 0x7fbdb80d0e10 2024-01-16T14:55:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.861+0000 7fbdb67fc700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 0x7fbda406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.861+0000 7fbdb67fc700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 0x7fbda406d810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8155b00 tx=0x7fbdac007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:31.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:31.865+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbda8054af0 con 0x7fbdb80d0e10 2024-01-16T14:55:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.121+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbdb8075480 con 0x7fbdb80d0e10 2024-01-16T14:55:32.104 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-16T14:55:32.121+0000 7fcb0b119700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-16T14:55:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.123+0000 7fbd9ffff700 1 -- 172.21.15.26:0/390913935 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbda8058110 con 0x7fbdb80d0e10 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 msgr2=0x7fbda406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 0x7fbda406d810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8155b00 tx=0x7fbdac007ec0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 msgr2=0x7fbdb81555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d800 tx=0x7fbda800db10 comp rx=0 tx=0).stop 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 shutdown_connections 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdb815e1e0 0x7fbdb81f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbda406b350 0x7fbda406d810 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8155b00 tx=0x7fbdac007ec0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb80d0e10 0x7fbdb81555c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 --2- 172.21.15.26:0/390913935 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbdb80d2810 0x7fbdb80b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.126+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 >> 172.21.15.26:0/390913935 conn(0x7fbdb80d3dd0 msgr2=0x7fbdb80b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.127+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 shutdown_connections 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.127+0000 7fbdbd83d700 1 -- 172.21.15.26:0/390913935 wait complete. 2024-01-16T14:55:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-16T14:55:32.110 INFO:tasks.ceph.mds.b.smithi093.stderr: -6855> 2024-01-16T14:54:38.802+0000 7f90228d5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-01-16T14:55:32.112 INFO:tasks.ceph.mds.b.smithi093.stderr: -5649> 2024-01-16T14:54:49.820+0000 7f901c8c9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T14:55:32.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-16T14:55:32.145 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2024-01-16T14:55:32.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 Processor -- start 2024-01-16T14:55:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 -- start start 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f07e806fe00 0x7f07e80cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07e80d2750 0x7f07e80d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80ceec0 con 0x7f07e815e190 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80cf110 con 0x7f07e80d2750 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.279+0000 7f07ed48b700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80cf3f0 con 0x7f07e806fe00 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07e80d2750 0x7f07e80d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07e80d2750 0x7f07e80d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50450/0 (socket says 172.21.15.26:50450) 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 -- 172.21.15.26:0/820078357 learned_addr learned my addr 172.21.15.26:0/820078357 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e67fc700 1 --2- 172.21.15.26:0/820078357 >> v2:172.21.15.93:3300/0 conn(0x7f07e806fe00 0x7f07e80cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 -- 172.21.15.26:0/820078357 >> v2:172.21.15.93:3300/0 conn(0x7f07e806fe00 msgr2=0x7f07e80cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 --2- 172.21.15.26:0/820078357 >> v2:172.21.15.93:3300/0 conn(0x7f07e806fe00 0x7f07e80cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 -- 172.21.15.26:0/820078357 >> v1:172.21.15.93:6789/0 conn(0x7f07e815e190 legacy=0x7f07e806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 -- 172.21.15.26:0/820078357 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80d0110 con 0x7f07e80d2750 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e77fe700 1 --2- 172.21.15.26:0/820078357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e80d2750 0x7f07e80d0630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f07e8162db0 tx=0x7f07e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8c435186ff8f96 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e5ffb700 1 -- 172.21.15.26:0/820078357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000e070 con 0x7f07e80d2750 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e5ffb700 1 -- 172.21.15.26:0/820078357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e000b9c0 con 0x7f07e80d2750 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.280+0000 7f07e5ffb700 1 -- 172.21.15.26:0/820078357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000bd70 con 0x7f07e80d2750 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 -- 172.21.15.26:0/820078357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e80d2750 msgr2=0x7f07e80d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 --2- 172.21.15.26:0/820078357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e80d2750 0x7f07e80d0630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f07e8162db0 tx=0x7f07e000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 -- 172.21.15.26:0/820078357 shutdown_connections 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 --2- 172.21.15.26:0/820078357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e80d2750 0x7f07e80d0630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 --2- 172.21.15.26:0/820078357 >> v2:172.21.15.93:3300/0 conn(0x7f07e806fe00 0x7f07e80cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 -- 172.21.15.26:0/820078357 >> 172.21.15.26:0/820078357 conn(0x7f07e80d1c30 msgr2=0x7f07e80d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 -- 172.21.15.26:0/820078357 shutdown_connections 2024-01-16T14:55:32.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.281+0000 7f07ed48b700 1 -- 172.21.15.26:0/820078357 wait complete. 2024-01-16T14:55:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 Processor -- start 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 -- start start 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e806fe00 0x7f07e81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e80d2750 0x7f07e80b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8153930 con 0x7f07e815e190 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e81593b0 con 0x7f07e806fe00 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.283+0000 7f07ed48b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80c9070 con 0x7f07e80d2750 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50454/0 (socket says 172.21.15.26:50454) 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 -- 172.21.15.26:0/2884691808 learned_addr learned my addr 172.21.15.26:0/2884691808 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e6ffd700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e806fe00 0x7f07e81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e67fc700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e80d2750 0x7f07e80b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 -- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e80d2750 msgr2=0x7f07e80b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e80d2750 0x7f07e80b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 -- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e806fe00 msgr2=0x7f07e81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e806fe00 0x7f07e81533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 -- 172.21.15.26:0/2884691808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0009b30 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07e77fe700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07e8072160 tx=0x7f07e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00099b0 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e00092e0 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e001b630 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e80c9350 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.284+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e80c9910 con 0x7f07e815e190 2024-01-16T14:55:32.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.285+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8075480 con 0x7f07e815e190 2024-01-16T14:55:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.286+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f07e001b790 con 0x7f07e815e190 2024-01-16T14:55:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.286+0000 7f07cf7fe700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 0x7f07d406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.286+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f07e008b450 con 0x7f07e815e190 2024-01-16T14:55:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.287+0000 7f07e6ffd700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 0x7f07d406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.287+0000 7f07e6ffd700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 0x7f07d406d810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cdac0 tx=0x7f07d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.291+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07e00562f0 con 0x7f07e815e190 2024-01-16T14:55:32.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:32.554+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07e8162a80 con 0x7f07e815e190 2024-01-16T14:55:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.124+0000 7f07cf7fe700 1 -- 172.21.15.26:0/2884691808 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07e0059910 con 0x7f07e815e190 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 msgr2=0x7f07d406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 0x7f07d406d810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cdac0 tx=0x7f07d8009040 comp rx=0 tx=0).stop 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 msgr2=0x7f07e80b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07e8072160 tx=0x7f07e0004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 shutdown_connections 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f07d406b350 0x7f07d406d810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07e815e190 0x7f07e80b5fc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e806fe00 0x7f07e81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 --2- 172.21.15.26:0/2884691808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e80d2750 0x7f07e80b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 >> 172.21.15.26:0/2884691808 conn(0x7f07e80d1c30 msgr2=0x7f07e80d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:33.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 shutdown_connections 2024-01-16T14:55:33.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.128+0000 7f07ed48b700 1 -- 172.21.15.26:0/2884691808 wait complete. 2024-01-16T14:55:33.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-16T14:55:33.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.289+0000 7fbcb7808700 1 Processor -- start 2024-01-16T14:55:33.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.290+0000 7fbcb7808700 1 -- start start 2024-01-16T14:55:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.290+0000 7fbcb7808700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbcb01692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.290+0000 7fbcb7808700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb006cb80 con 0x7fbcb006ef30 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.290+0000 7fbcb7808700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb00cf9e0 con 0x7fbcb006c7a0 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.290+0000 7fbcb7808700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00cf100 con 0x7fbcb0166ef0 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbcb4da3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbcb006ef30 0x7fbcb00ce9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41076/0 (socket says 172.21.15.26:41076) 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbcb4da3700 1 -- 172.21.15.26:0/721503506 learned_addr learned my addr 172.21.15.26:0/721503506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbcb5da5700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbcb01692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646750399 0 0) 0x7fbcb006cb80 con 0x7fbcb006ef30 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc9c0035b0 con 0x7fbcb006ef30 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3910772718 0 0) 0x7fbc9c0035b0 con 0x7fbcb006ef30 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 msgr2=0x7fbcb01692e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbcb01692e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 >> v1:172.21.15.93:6789/0 conn(0x7fbcb006c7a0 legacy=0x7fbcb00ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb006d610 con 0x7fbcb006ef30 2024-01-16T14:55:33.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7fbcac001670 con 0x7fbcb006ef30 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 >> v1:172.21.15.26:6789/0 conn(0x7fbcb006ef30 legacy=0x7fbcb00ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc9c003a30 0x7fbc9c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c0043e0 0x7fbc9c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbc9c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00cf100 con 0x7fbc9c003a30 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c00bac0 con 0x7fbc9c0043e0 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.291+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c00bc80 con 0x7fbcb0166ef0 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc9c003a30 0x7fbc9c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb4da3700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c0043e0 0x7fbc9c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb5da5700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbc9c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 -- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 msgr2=0x7fbc9c0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb0166ef0 0x7fbc9c0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 -- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c0043e0 msgr2=0x7fbc9c0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c0043e0 0x7fbc9c0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 -- 172.21.15.26:0/721503506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0035b0 con 0x7fbc9c003a30 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc9c003a30 0x7fbc9c003e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbca00018d0 tx=0x7fbc9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177fca86af93f9c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000c690 con 0x7fbc9c003a30 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000c810 con 0x7fbc9c003a30 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.292+0000 7fbca7fff700 1 -- 172.21.15.26:0/721503506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000cbc0 con 0x7fbc9c003a30 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 -- 172.21.15.26:0/721503506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc9c003a30 msgr2=0x7fbc9c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 --2- 172.21.15.26:0/721503506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc9c003a30 0x7fbc9c003e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbca00018d0 tx=0x7fbc9c005880 comp rx=0 tx=0).stop 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb5da5700 1 -- 172.21.15.26:0/721503506 reap_dead start 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 -- 172.21.15.26:0/721503506 shutdown_connections 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 -- 172.21.15.26:0/721503506 >> 172.21.15.26:0/721503506 conn(0x7fbcb00ccee0 msgr2=0x7fbcb00cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 -- 172.21.15.26:0/721503506 shutdown_connections 2024-01-16T14:55:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.293+0000 7fbcb7808700 1 -- 172.21.15.26:0/721503506 wait complete. 2024-01-16T14:55:33.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.294+0000 7fbcb7808700 1 Processor -- start 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.294+0000 7fbcb7808700 1 -- start start 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 0x7fbcb0166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 0x7fbcb01f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb01f49b0 con 0x7fbcb0154290 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb0154c60 con 0x7fbc9c003a30 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb7808700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb014d4b0 con 0x7fbcb00ad860 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50480/0 (socket says 172.21.15.26:50480) 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 -- 172.21.15.26:0/1775408133 learned_addr learned my addr 172.21.15.26:0/1775408133 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 0x7fbcb0166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb4da3700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 0x7fbcb01f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 -- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 msgr2=0x7fbcb01f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 0x7fbcb01f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 -- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 msgr2=0x7fbcb0166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 0x7fbcb0166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb5da5700 1 -- 172.21.15.26:0/1775408133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb01f7410 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 0x7fbcb0166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.295+0000 7fbcb4da3700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 0x7fbcb01f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbcb5da5700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0155670 tx=0x7fbca800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800e040 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800f040 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb01f76f0 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800fc60 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.296+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb01f7cb0 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.297+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb0068670 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.298+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbca8004030 con 0x7fbcb0154290 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.298+0000 7fbca5ffb700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 0x7fbc9406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.298+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbca808a210 con 0x7fbcb0154290 2024-01-16T14:55:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.298+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 0x7fbc9406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.299+0000 7fbcb55a4700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 0x7fbc9406d810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbca0014040 tx=0x7fbca0005040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:33.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.303+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbca80550b0 con 0x7fbcb0154290 2024-01-16T14:55:33.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:33.514+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbcb00cde10 con 0x7fbcb0154290 2024-01-16T14:55:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.129+0000 7fbca5ffb700 1 -- 172.21.15.26:0/1775408133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbca80586d0 con 0x7fbcb0154290 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 msgr2=0x7fbc9406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 0x7fbc9406d810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbca0014040 tx=0x7fbca0005040 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 msgr2=0x7fbcb0154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0155670 tx=0x7fbca800f490 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 shutdown_connections 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbc9406b350 0x7fbc9406d810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcb0154290 0x7fbcb0154720 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc9c003a30 0x7fbcb0166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 --2- 172.21.15.26:0/1775408133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb00ad860 0x7fbcb01f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 >> 172.21.15.26:0/1775408133 conn(0x7fbcb00ccee0 msgr2=0x7fbcb0071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 shutdown_connections 2024-01-16T14:55:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.132+0000 7fbcb7808700 1 -- 172.21.15.26:0/1775408133 wait complete. 2024-01-16T14:55:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-01-16T14:55:34.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-16T14:55:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.296+0000 7f07130b2700 1 Processor -- start 2024-01-16T14:55:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.297+0000 7f07130b2700 1 -- start start 2024-01-16T14:55:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f07130b2700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f070c1453e0 0x7f070c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f07130b2700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c06b6e0 0x7f070c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f07130b2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c0cf940 con 0x7f070c1457c0 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f07130b2700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c0cfc20 con 0x7f070c1453e0 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f07130b2700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0cff00 con 0x7f070c06b6e0 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f0710e4e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f070c1453e0 0x7f070c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f070bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f070c1457c0 0x7f070c06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41090/0 (socket says 172.21.15.26:41090) 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f070bfff700 1 -- 172.21.15.26:0/2898084534 learned_addr learned my addr 172.21.15.26:0/2898084534 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f071164f700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c06b6e0 0x7f070c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.298+0000 7f0710e4e700 1 -- 172.21.15.26:0/2898084534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c06b6e0 msgr2=0x7f070c06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f0710e4e700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c06b6e0 0x7f070c06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f0710e4e700 1 -- 172.21.15.26:0/2898084534 >> v1:172.21.15.26:6789/0 conn(0x7f070c1457c0 legacy=0x7f070c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f0710e4e700 1 -- 172.21.15.26:0/2898084534 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c0d01e0 con 0x7f070c1453e0 2024-01-16T14:55:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f070b7fe700 1 -- 172.21.15.26:0/2898084534 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139470270 0 0) 0x7f070c0cf940 con 0x7f070c1457c0 2024-01-16T14:55:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f0710e4e700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1453e0 0x7f070c0ceff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0700009840 tx=0x7f0700009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c78fe09252e845 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f070b7fe700 1 -- 172.21.15.26:0/2898084534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011830 con 0x7f070c1453e0 2024-01-16T14:55:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f070b7fe700 1 -- 172.21.15.26:0/2898084534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700011990 con 0x7f070c1453e0 2024-01-16T14:55:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f070b7fe700 1 -- 172.21.15.26:0/2898084534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011d20 con 0x7f070c1453e0 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f07130b2700 1 -- 172.21.15.26:0/2898084534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1453e0 msgr2=0x7f070c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.299+0000 7f07130b2700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1453e0 0x7f070c0ceff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0700009840 tx=0x7f0700009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 -- 172.21.15.26:0/2898084534 shutdown_connections 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c06b6e0 0x7f070c06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 --2- 172.21.15.26:0/2898084534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1453e0 0x7f070c0ceff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 -- 172.21.15.26:0/2898084534 >> 172.21.15.26:0/2898084534 conn(0x7f070c0b2840 msgr2=0x7f070c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 -- 172.21.15.26:0/2898084534 shutdown_connections 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.300+0000 7f07130b2700 1 -- 172.21.15.26:0/2898084534 wait complete. 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.301+0000 7f07130b2700 1 Processor -- start 2024-01-16T14:55:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.301+0000 7f07130b2700 1 -- start start 2024-01-16T14:55:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f070c06b6e0 0x7f070c140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c1453e0 0x7f070c15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c13cf20 con 0x7f070c1457c0 2024-01-16T14:55:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c141300 con 0x7f070c06b6e0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f07130b2700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c13d170 con 0x7f070c1453e0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50486/0 (socket says 172.21.15.26:50486) 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 -- 172.21.15.26:0/1706598313 learned_addr learned my addr 172.21.15.26:0/1706598313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f070bfff700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c1453e0 0x7f070c15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f0710e4e700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f070c06b6e0 0x7f070c140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 -- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c1453e0 msgr2=0x7f070c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c1453e0 0x7f070c15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 -- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f070c06b6e0 msgr2=0x7f070c140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f070c06b6e0 0x7f070c140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.302+0000 7f071164f700 1 -- 172.21.15.26:0/1706598313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000094a0 con 0x7f070c1457c0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f071164f700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f070400ef90 tx=0x7f070400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400cc90 con 0x7f070c1457c0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070400f040 con 0x7f070c1457c0 2024-01-16T14:55:34.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c13d450 con 0x7f070c1457c0 2024-01-16T14:55:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704014480 con 0x7f070c1457c0 2024-01-16T14:55:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.303+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c13da10 con 0x7f070c1457c0 2024-01-16T14:55:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.304+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c13dc30 con 0x7f070c1457c0 2024-01-16T14:55:34.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.304+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f07040145e0 con 0x7f070c1457c0 2024-01-16T14:55:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.305+0000 7f07097fa700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 0x7f06f006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.305+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f070408a600 con 0x7f070c1457c0 2024-01-16T14:55:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.305+0000 7f0710e4e700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 0x7f06f006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.306+0000 7f0710e4e700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 0x7f06f006d810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0700009fd0 tx=0x7f0700008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:34.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.309+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07040555e0 con 0x7f070c1457c0 2024-01-16T14:55:34.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:34.522+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f070c068670 con 0x7f070c1457c0 2024-01-16T14:55:35.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.130+0000 7f07097fa700 1 -- 172.21.15.26:0/1706598313 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f07040180d0 con 0x7f070c1457c0 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 msgr2=0x7f06f006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 0x7f06f006d810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0700009fd0 tx=0x7f0700008030 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 msgr2=0x7f070c0ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f070400ef90 tx=0x7f070400c3a0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 shutdown_connections 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f006b350 0x7f06f006d810 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0700009fd0 tx=0x7f0700008030 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070c1457c0 0x7f070c0ac2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f070c06b6e0 0x7f070c140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 --2- 172.21.15.26:0/1706598313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f070c1453e0 0x7f070c15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 >> 172.21.15.26:0/1706598313 conn(0x7f070c0b2840 msgr2=0x7f070c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.134+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 shutdown_connections 2024-01-16T14:55:35.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.135+0000 7f07130b2700 1 -- 172.21.15.26:0/1706598313 wait complete. 2024-01-16T14:55:35.119 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-01-16T14:55:35.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-16T14:55:35.137 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-16T14:55:35.137 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-16T14:55:35.137 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T14:55:35.137 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-16T14:55:35.137 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-16T14:55:35.137 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-16T14:55:35.138 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T14:55:35.138 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-16T14:55:35.138 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-16T14:55:35.138 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-16T14:55:35.138 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T14:55:35.138 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-16T14:55:35.138 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-16T14:55:35.138 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-16T14:55:35.139 INFO:teuthology.orchestra.run:waiting for 300 2024-01-16T14:55:35.148 INFO:tasks.ceph.mds.a:Stopped 2024-01-16T14:55:35.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-16T14:55:35.149 INFO:tasks.ceph.mds.c:Stopped 2024-01-16T14:55:35.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-16T14:55:35.150 INFO:tasks.ceph.mds.d:Stopped 2024-01-16T14:55:35.150 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-16T14:55:35.151 INFO:tasks.ceph.mds.b:Stopped 2024-01-16T14:55:35.151 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-16T14:55:35.191 INFO:tasks.ceph.mds.a:Started 2024-01-16T14:55:35.191 DEBUG:teuthology.parallel:result is None 2024-01-16T14:55:35.192 INFO:tasks.ceph.mds.c:Started 2024-01-16T14:55:35.192 DEBUG:teuthology.parallel:result is None 2024-01-16T14:55:35.193 INFO:tasks.ceph.mds.d:Started 2024-01-16T14:55:35.193 DEBUG:teuthology.parallel:result is None 2024-01-16T14:55:35.194 INFO:tasks.ceph.mds.b:Started 2024-01-16T14:55:35.194 DEBUG:teuthology.parallel:result is None 2024-01-16T14:55:35.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-16T14:55:35.280 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2024-01-16T14:55:35.280 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2024-01-16T14:55:35.286 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-16T14:55:35.286 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-16T14:55:35.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.360+0000 7fabf31e1700 1 Processor -- start 2024-01-16T14:55:35.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.360+0000 7fabf31e1700 1 -- start start 2024-01-16T14:55:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 0x7fabec0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 0x7fabec0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec06f610 con 0x7fabec0cef20 2024-01-16T14:55:35.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0cf370 con 0x7fabec0ce560 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf31e1700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0cd7f0 con 0x7fabec15c010 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabebfff700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 0x7fabec0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf177e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 0x7fabec0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:55996/0 (socket says 172.21.15.26:55996) 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabebfff700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 0x7fabec0ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:44508/0 (socket says 172.21.15.26:44508) 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 -- 172.21.15.26:0/4085167490 learned_addr learned my addr 172.21.15.26:0/4085167490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 -- 172.21.15.26:0/4085167490 >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 msgr2=0x7fabec0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/4085167490 >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 0x7fabec0ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 -- 172.21.15.26:0/4085167490 >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 msgr2=0x7fabec0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/4085167490 >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 0x7fabec0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 -- 172.21.15.26:0/4085167490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0cdac0 con 0x7fabec15c010 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf177e700 1 --2- 172.21.15.26:0/4085167490 >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 0x7fabec0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.361+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/4085167490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabec072370 tx=0x7fabe0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af6c0382fb0e78e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabeb7fe700 1 -- 172.21.15.26:0/4085167490 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0011830 con 0x7fabec15c010 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabeb7fe700 1 -- 172.21.15.26:0/4085167490 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0011990 con 0x7fabec15c010 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabeb7fe700 1 -- 172.21.15.26:0/4085167490 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0011d20 con 0x7fabec15c010 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 -- 172.21.15.26:0/4085167490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 msgr2=0x7fabec0cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 --2- 172.21.15.26:0/4085167490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabec072370 tx=0x7fabe0009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 -- 172.21.15.26:0/4085167490 shutdown_connections 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 --2- 172.21.15.26:0/4085167490 >> v2:172.21.15.26:3300/0 conn(0x7fabec0cef20 0x7fabec0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 --2- 172.21.15.26:0/4085167490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec15c010 0x7fabec0cd2b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 --2- 172.21.15.26:0/4085167490 >> v2:172.21.15.93:3300/0 conn(0x7fabec0ce560 0x7fabec0ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 -- 172.21.15.26:0/4085167490 >> 172.21.15.26:0/4085167490 conn(0x7fabec0b3790 msgr2=0x7fabec0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 -- 172.21.15.26:0/4085167490 shutdown_connections 2024-01-16T14:55:35.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.362+0000 7fabf31e1700 1 -- 172.21.15.26:0/4085167490 wait complete. 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.364+0000 7fabf31e1700 1 Processor -- start 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.364+0000 7fabf31e1700 1 -- start start 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0cef20 0x7fabec15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabec15c010 0x7fabec0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1f5aa0 con 0x7fabec0ce560 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec159890 con 0x7fabec15c010 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf31e1700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0c5730 con 0x7fabec0cef20 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36592/0 (socket says 172.21.15.26:36592) 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 -- 172.21.15.26:0/505322688 learned_addr learned my addr 172.21.15.26:0/505322688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabebfff700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0cef20 0x7fabec15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf177e700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabec15c010 0x7fabec0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 -- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0cef20 msgr2=0x7fabec15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0cef20 0x7fabec15a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 -- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabec15c010 msgr2=0x7fabec0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabec15c010 0x7fabec0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 -- 172.21.15.26:0/505322688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4009a70 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.365+0000 7fabf0f7d700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009c10 tx=0x7fabe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.366+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0004450 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.366+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe001e070 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.366+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe0009550 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.366+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe00038e0 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.366+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec1f4fc0 con 0x7fabec0ce560 2024-01-16T14:55:35.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.367+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec075480 con 0x7fabec0ce560 2024-01-16T14:55:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.367+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fabe0003a40 con 0x7fabec0ce560 2024-01-16T14:55:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.368+0000 7fabe97fa700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 0x7fabd006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.368+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fabe008a530 con 0x7fabec0ce560 2024-01-16T14:55:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.368+0000 7fabebfff700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 0x7fabd006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:35.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.369+0000 7fabebfff700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 0x7fabd006d810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabec0d1610 tx=0x7fabd8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:35.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.373+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabe0055730 con 0x7fabec0ce560 2024-01-16T14:55:35.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:35.602+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fabec0b7590 con 0x7fabec0ce560 2024-01-16T14:55:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.146+0000 7fabe97fa700 1 -- 172.21.15.26:0/505322688 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7fabe0020080 con 0x7fabec0ce560 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 msgr2=0x7fabd006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 0x7fabd006d810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fabec0d1610 tx=0x7fabd8009210 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 msgr2=0x7fabec15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009c10 tx=0x7fabe0004750 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 shutdown_connections 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabec15c010 0x7fabec0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fabd006b350 0x7fabd006d810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabec0ce560 0x7fabec15a160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 --2- 172.21.15.26:0/505322688 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0cef20 0x7fabec15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 >> 172.21.15.26:0/505322688 conn(0x7fabec0b3790 msgr2=0x7fabec0b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 shutdown_connections 2024-01-16T14:55:36.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.150+0000 7fabf31e1700 1 -- 172.21.15.26:0/505322688 wait complete. 2024-01-16T14:55:36.134 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-16T14:55:36.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-16T14:55:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.307+0000 7fd914632700 1 Processor -- start 2024-01-16T14:55:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.307+0000 7fd914632700 1 -- start start 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd914632700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd90c0b4f20 0x7fd90c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd914632700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c072760 con 0x7fd90c0722d0 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd914632700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c06e550 con 0x7fd90c15abb0 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd914632700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c06f3b0 con 0x7fd90c0b4f20 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd912bcf700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd90c0722d0 0x7fd90c0cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53818/0 (socket says 172.21.15.26:53818) 2024-01-16T14:55:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd912bcf700 1 -- 172.21.15.26:0/2275172807 learned_addr learned my addr 172.21.15.26:0/2275172807 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd911bcd700 1 --2- 172.21.15.26:0/2275172807 >> v2:172.21.15.93:3300/0 conn(0x7fd90c0b4f20 0x7fd90c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719368958 0 0) 0x7fd904001490 con 0x7fd90c0722d0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.308+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f80035b0 con 0x7fd90c0722d0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4053156499 0 0) 0x7fd8f80035b0 con 0x7fd90c0722d0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 >> v2:172.21.15.93:3300/0 conn(0x7fd90c0b4f20 msgr2=0x7fd90c071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 --2- 172.21.15.26:0/2275172807 >> v2:172.21.15.93:3300/0 conn(0x7fd90c0b4f20 0x7fd90c071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 >> v1:172.21.15.93:6789/0 conn(0x7fd90c15abb0 legacy=0x7fd90c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c06e7a0 con 0x7fd90c0722d0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7fd904001b80 con 0x7fd90c0722d0 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd911bcd700 1 --2- 172.21.15.26:0/2275172807 >> v2:172.21.15.93:3300/0 conn(0x7fd90c0b4f20 0x7fd90c071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 >> v1:172.21.15.26:6789/0 conn(0x7fd90c0722d0 legacy=0x7fd90c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 0x7fd8f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8f8004400 0x7fd8f80048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f80050d0 0x7fd8f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c06f3b0 con 0x7fd8f8003a00 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f800c030 con 0x7fd8f80050d0 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f800e400 con 0x7fd8f8004400 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 0x7fd8f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd911bcd700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8f8004400 0x7fd8f80048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd912bcf700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f80050d0 0x7fd8f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 -- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8f8004400 msgr2=0x7fd8f80048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8f8004400 0x7fd8f80048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 -- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f80050d0 msgr2=0x7fd8f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f80050d0 0x7fd8f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd912bcf700 1 -- 172.21.15.26:0/2275172807 reap_dead start 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 -- 172.21.15.26:0/2275172807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80035b0 con 0x7fd8f8003a00 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.309+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 0x7fd8f8003de0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9080018d0 tx=0x7fd8f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ab11744f64bfb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080052d0 con 0x7fd8f8003a00 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908005430 con 0x7fd8f8003a00 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd9113cc700 1 -- 172.21.15.26:0/2275172807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080038c0 con 0x7fd8f8003a00 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 -- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 msgr2=0x7fd8f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 0x7fd8f8003de0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9080018d0 tx=0x7fd8f8005880 comp rx=0 tx=0).stop 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 -- 172.21.15.26:0/2275172807 shutdown_connections 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 --2- 172.21.15.26:0/2275172807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8f8003a00 0x7fd8f8003de0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 -- 172.21.15.26:0/2275172807 >> 172.21.15.26:0/2275172807 conn(0x7fd90c0d0aa0 msgr2=0x7fd90c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 -- 172.21.15.26:0/2275172807 shutdown_connections 2024-01-16T14:55:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.310+0000 7fd914632700 1 -- 172.21.15.26:0/2275172807 wait complete. 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.312+0000 7fd914632700 1 Processor -- start 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.312+0000 7fd914632700 1 -- start start 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.312+0000 7fd914632700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f8003a00 0x7fd90c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd90c0af5e0 0x7fd90c0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c14e330 con 0x7fd90c0cf430 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0cf8b0 con 0x7fd8f8003a00 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0af150 con 0x7fd90c0af5e0 2024-01-16T14:55:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36612/0 (socket says 172.21.15.26:36612) 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 -- 172.21.15.26:0/2571907604 learned_addr learned my addr 172.21.15.26:0/2571907604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd911bcd700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd90c0af5e0 0x7fd90c0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f8003a00 0x7fd90c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 -- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd90c0af5e0 msgr2=0x7fd90c0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd90c0af5e0 0x7fd90c0ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 -- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f8003a00 msgr2=0x7fd90c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f8003a00 0x7fd90c14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 -- 172.21.15.26:0/2571907604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c14ab30 con 0x7fd90c0cf430 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd912bcf700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd90c072f80 tx=0x7fd904007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904007980 con 0x7fd90c0cf430 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd904007ae0 con 0x7fd90c0cf430 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0aeb20 con 0x7fd90c0cf430 2024-01-16T14:55:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904013630 con 0x7fd90c0cf430 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.313+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c0ae7e0 con 0x7fd90c0cf430 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.314+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c0af2e0 con 0x7fd90c0cf430 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.315+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd904013790 con 0x7fd90c0cf430 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.316+0000 7fd902ffd700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 0x7fd8f006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.316+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd90408a900 con 0x7fd90c0cf430 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.316+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 0x7fd8f006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.317+0000 7fd9123ce700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 0x7fd8f006d810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd908005000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.320+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd904055aa0 con 0x7fd90c0cf430 2024-01-16T14:55:36.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.615+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd90c072760 con 0x7fd90c0cf430 2024-01-16T14:55:36.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.615+0000 7fd902ffd700 1 -- 172.21.15.26:0/2571907604 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd90c072760 con 0x7fd90c0cf430 2024-01-16T14:55:36.598 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:36.598 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:55:36.598 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-16T14:55:36.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZl4XAuHhAA4D3kwEeG+gbjNF4/rF0U2Q==", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZlIT6wLBAASm0XUVaPtvstSdGMyfPAOA==", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZltK62JhAArIT8N0ucUNkEhlpCFndkOA==", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZlDRx6NBAA+uk0U+typ/+LV+BO8LHnYg==", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCFmKZlyKfdGxAATOUoliQPcYm2vSvVcBZ9rw==", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCImKZlUTczGxAAyhDeEJ1jrpRNOqtq4YzKFw==", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCLmKZlaAhbGhAAdBk7IP19PAqvO7FzA6OjHA==", 2024-01-16T14:55:36.602 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCOmKZlXT5FGRAAIXAbV/zGy0mu1hML6KdHfg==", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCUmKZlf9aQHhAAfHJuQ+qPwhvpFTO3LNDxgA==", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCXmKZl+MVTHhAAtNjpAJRt0UluZ0W7ulYcug==", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCamKZlQAW8HRAADpM2I3F32HIjlkmeIBxU9g==", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCdmKZlo+HAGxAA4nE/koKFEvlcqHZAeibWcg==", 2024-01-16T14:55:36.605 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZl8JqJORAAXPhIkgN/hhjJtdjFgPeRLQ==", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCEmKZlTU9/BRAA3FQuMz/iXQtANu603C0hDQ==", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCEmKZlQ4foARAAdLlG4GlY3XsbaeZnnhsqdQ==", 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCEmKZl8AnOCBAAaUevXdwBXt8tPHjFBxhu6A==", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-16T14:55:36.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCCmKZlawH4KBAAATyBvLprUgOcG6v1M4/oRA==", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlrLsGJBAAwAmw+gHRLAbu9dUxRSei+Q==", 2024-01-16T14:55:36.609 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlcMkGJBAAfs/dOdYr8znu6yfkjs92MQ==", 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlH9MGJBAAIGWpbefbcg1uKJhstwiIAA==", 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2024-01-16T14:55:36.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlPtwGJBAAsIkk4O7WWfj2dCksZlLlTw==", 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlXOUGJBAALyjBCwrQi3DZGR7Csctd6A==", 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCjmKZlrO4GJBAAnpjXCIYN1q7qSt5oJKdhAg==", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZlJLDfGRAAK1YRyL4/DNnMX0M7WJThLA==", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQCDmKZlBayOFRAA53grUUs3v0uIUgKWz5ep3Q==", 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-16T14:55:36.613 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.617+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 msgr2=0x7fd8f006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.617+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 0x7fd8f006d810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd908000c00 tx=0x7fd908005000 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.617+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 msgr2=0x7fd90c0ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.617+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd90c072f80 tx=0x7fd904007340 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 shutdown_connections 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd8f006b350 0x7fd8f006d810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd90c0cf430 0x7fd90c0ad860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8f8003a00 0x7fd90c14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 --2- 172.21.15.26:0/2571907604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd90c0af5e0 0x7fd90c0ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 >> 172.21.15.26:0/2571907604 conn(0x7fd90c0d0aa0 msgr2=0x7fd90c0b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 shutdown_connections 2024-01-16T14:55:36.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.618+0000 7fd914632700 1 -- 172.21.15.26:0/2571907604 wait complete. 2024-01-16T14:55:36.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-16T14:55:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.771+0000 7fd337e82700 1 Processor -- start 2024-01-16T14:55:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.771+0000 7fd337e82700 1 -- start start 2024-01-16T14:55:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd337e82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3300726a0 0x7fd330072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd337e82700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd337e82700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33015f3e0 con 0x7fd33006bac0 2024-01-16T14:55:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd337e82700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300affd0 con 0x7fd3300726a0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd337e82700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330073500 con 0x7fd3300b3d90 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3300726a0 0x7fd330072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3300726a0 0x7fd330072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36622/0 (socket says 172.21.15.26:36622) 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 -- 172.21.15.26:0/3009383659 learned_addr learned my addr 172.21.15.26:0/3009383659 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd33541d700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 -- 172.21.15.26:0/3009383659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 msgr2=0x7fd330072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 -- 172.21.15.26:0/3009383659 >> v1:172.21.15.93:6789/0 conn(0x7fd33006bac0 legacy=0x7fd33006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.772+0000 7fd335c1e700 1 -- 172.21.15.26:0/3009383659 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33015e7d0 con 0x7fd3300726a0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd335c1e700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd330072a80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098f0 tx=0x7fd32c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad89f8ddc43e07b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd334c1c700 1 -- 172.21.15.26:0/3009383659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c011830 con 0x7fd3300726a0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd334c1c700 1 -- 172.21.15.26:0/3009383659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c011990 con 0x7fd3300726a0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd334c1c700 1 -- 172.21.15.26:0/3009383659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c011d20 con 0x7fd3300726a0 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 -- 172.21.15.26:0/3009383659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 msgr2=0x7fd330072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd330072a80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098f0 tx=0x7fd32c009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 -- 172.21.15.26:0/3009383659 shutdown_connections 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd330072a80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 --2- 172.21.15.26:0/3009383659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.773+0000 7fd337e82700 1 -- 172.21.15.26:0/3009383659 >> 172.21.15.26:0/3009383659 conn(0x7fd3300717f0 msgr2=0x7fd3300b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.774+0000 7fd337e82700 1 -- 172.21.15.26:0/3009383659 shutdown_connections 2024-01-16T14:55:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.774+0000 7fd337e82700 1 -- 172.21.15.26:0/3009383659 wait complete. 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.775+0000 7fd337e82700 1 Processor -- start 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.775+0000 7fd337e82700 1 -- start start 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd33009b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33009bd00 con 0x7fd3300726a0 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33009ad20 con 0x7fd33006bac0 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd337e82700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330147930 con 0x7fd3300b3d90 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd335c1e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd335c1e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:44572/0 (socket says 172.21.15.26:44572) 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd335c1e700 1 -- 172.21.15.26:0/477429495 learned_addr learned my addr 172.21.15.26:0/477429495 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 -- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 msgr2=0x7fd330158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 -- 172.21.15.26:0/477429495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 msgr2=0x7fd33009b7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd33009b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.776+0000 7fd33641f700 1 -- 172.21.15.26:0/477429495 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c009550 con 0x7fd3300b3d90 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd335c1e700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd33641f700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330145eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd32800cb10 tx=0x7fd32800ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328003c00 con 0x7fd3300b3d90 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd328003d60 con 0x7fd3300b3d90 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33009afc0 con 0x7fd3300b3d90 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800aaa0 con 0x7fd3300b3d90 2024-01-16T14:55:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.777+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd330147c10 con 0x7fd3300b3d90 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.778+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33009b4e0 con 0x7fd3300b3d90 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.779+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd32800ac00 con 0x7fd3300b3d90 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.779+0000 7fd3227fc700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 0x7fd31406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.779+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd328089750 con 0x7fd3300b3d90 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.779+0000 7fd335c1e700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 0x7fd31406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.780+0000 7fd335c1e700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 0x7fd31406d810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009c40 tx=0x7fd32c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:36.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:36.784+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd33009b4e0 con 0x7fd3300b3d90 2024-01-16T14:55:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.080+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd33009b4e0 con 0x7fd3300b3d90 2024-01-16T14:55:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.086+0000 7fd3227fc700 1 -- 172.21.15.26:0/477429495 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd33009b4e0 con 0x7fd3300b3d90 2024-01-16T14:55:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 msgr2=0x7fd31406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 0x7fd31406d810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009c40 tx=0x7fd32c008030 comp rx=0 tx=0).stop 2024-01-16T14:55:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 msgr2=0x7fd330145eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330145eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd32800cb10 tx=0x7fd32800ce20 comp rx=0 tx=0).stop 2024-01-16T14:55:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 shutdown_connections 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3300b3d90 0x7fd330145eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd33006bac0 0x7fd330158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd31406b350 0x7fd31406d810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 --2- 172.21.15.26:0/477429495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3300726a0 0x7fd33009b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 >> 172.21.15.26:0/477429495 conn(0x7fd3300717f0 msgr2=0x7fd3300ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 shutdown_connections 2024-01-16T14:55:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.088+0000 7fd337e82700 1 -- 172.21.15.26:0/477429495 wait complete. 2024-01-16T14:55:37.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-16T14:55:37.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.241+0000 7f2a5d738700 1 Processor -- start 2024-01-16T14:55:37.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.241+0000 7f2a5d738700 1 -- start start 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.241+0000 7f2a5d738700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2a5806fe00 0x7f2a58168db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a5d738700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580d10f0 con 0x7f2a5815e350 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a5d738700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580d0850 con 0x7f2a5815df70 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a5d738700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a580d0b30 con 0x7f2a5806fe00 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a567fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a5815e350 0x7f2a5806f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53828/0 (socket says 172.21.15.26:53828) 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a567fc700 1 -- 172.21.15.26:0/610814475 learned_addr learned my addr 172.21.15.26:0/610814475 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a577fe700 1 --2- 172.21.15.26:0/610814475 >> v2:172.21.15.93:3300/0 conn(0x7f2a5806fe00 0x7f2a58168db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367403851 0 0) 0x7f2a580d10f0 con 0x7f2a5815e350 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a400035b0 con 0x7f2a5815e350 2024-01-16T14:55:37.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865482878 0 0) 0x7f2a580d0850 con 0x7f2a5815df70 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a580d10f0 con 0x7f2a5815df70 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a577fe700 1 -- 172.21.15.26:0/610814475 >> v1:172.21.15.93:6789/0 conn(0x7f2a5815df70 legacy=0x7f2a5806eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403376730 0 0) 0x7f2a400035b0 con 0x7f2a5815e350 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a577fe700 1 -- 172.21.15.26:0/610814475 >> v1:172.21.15.26:6789/0 conn(0x7f2a5815e350 legacy=0x7f2a5806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a577fe700 1 -- 172.21.15.26:0/610814475 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580d0e10 con 0x7f2a5806fe00 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.242+0000 7f2a577fe700 1 --2- 172.21.15.26:0/610814475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5806fe00 0x7f2a58168db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009d90 tx=0x7f2a5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336c8dcde9f21f98 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5000e070 con 0x7f2a5806fe00 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50003a20 con 0x7f2a5806fe00 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a55ffb700 1 -- 172.21.15.26:0/610814475 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c001670 con 0x7f2a5806fe00 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 -- 172.21.15.26:0/610814475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5806fe00 msgr2=0x7f2a58168db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 --2- 172.21.15.26:0/610814475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5806fe00 0x7f2a58168db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009d90 tx=0x7f2a5000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 -- 172.21.15.26:0/610814475 shutdown_connections 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 --2- 172.21.15.26:0/610814475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5806fe00 0x7f2a58168db0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 -- 172.21.15.26:0/610814475 >> 172.21.15.26:0/610814475 conn(0x7f2a580b3720 msgr2=0x7f2a580b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:37.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 -- 172.21.15.26:0/610814475 shutdown_connections 2024-01-16T14:55:37.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.243+0000 7f2a5d738700 1 -- 172.21.15.26:0/610814475 wait complete. 2024-01-16T14:55:37.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.245+0000 7f2a5d738700 1 Processor -- start 2024-01-16T14:55:37.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.245+0000 7f2a5d738700 1 -- start start 2024-01-16T14:55:37.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.245+0000 7f2a5d738700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 0x7f2a581f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.245+0000 7f2a5d738700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 0x7f2a580c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a5d738700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a5d738700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580c8580 con 0x7f2a5815e350 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a5d738700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580c8f80 con 0x7f2a5815df70 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a5d738700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580c71f0 con 0x7f2a5806fe00 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36624/0 (socket says 172.21.15.26:36624) 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 -- 172.21.15.26:0/3447335229 learned_addr learned my addr 172.21.15.26:0/3447335229 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a56ffd700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 0x7f2a581f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a567fc700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 0x7f2a580c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 -- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 msgr2=0x7f2a581f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 0x7f2a581f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 -- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 msgr2=0x7f2a580c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 0x7f2a580c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 -- 172.21.15.26:0/3447335229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a50009a70 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a56ffd700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 0x7f2a581f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a567fc700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 0x7f2a580c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a577fe700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000b370 tx=0x7f2a50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500045c0 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50004720 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a580c7900 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.246+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5001d470 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.247+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a580c7ec0 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.247+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58075480 con 0x7f2a5815e350 2024-01-16T14:55:37.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.248+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2a5001d5d0 con 0x7f2a5815e350 2024-01-16T14:55:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.248+0000 7f2a3f7fe700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 0x7f2a4406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.249+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a5008b030 con 0x7f2a5815e350 2024-01-16T14:55:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.249+0000 7f2a56ffd700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 0x7f2a4406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.249+0000 7f2a56ffd700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 0x7f2a4406d810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a480011e0 tx=0x7f2a48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.253+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a50059d10 con 0x7f2a5815e350 2024-01-16T14:55:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.553+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2a580cf0b0 con 0x7f2a5815e350 2024-01-16T14:55:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.556+0000 7f2a3f7fe700 1 -- 172.21.15.26:0/3447335229 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2a500598a0 con 0x7f2a5815e350 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 msgr2=0x7f2a4406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 0x7f2a4406d810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a480011e0 tx=0x7f2a48009040 comp rx=0 tx=0).stop 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 msgr2=0x7f2a5806c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000b370 tx=0x7f2a50004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 shutdown_connections 2024-01-16T14:55:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a4406b350 0x7f2a4406d810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5815e350 0x7f2a5806c880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a5806fe00 0x7f2a581f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 --2- 172.21.15.26:0/3447335229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a5815df70 0x7f2a580c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 >> 172.21.15.26:0/3447335229 conn(0x7f2a580b3720 msgr2=0x7f2a5815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 shutdown_connections 2024-01-16T14:55:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.559+0000 7f2a5d738700 1 -- 172.21.15.26:0/3447335229 wait complete. 2024-01-16T14:55:37.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-16T14:55:37.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.710+0000 7ff1989e7700 1 Processor -- start 2024-01-16T14:55:37.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.710+0000 7ff1989e7700 1 -- start start 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff1989e7700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff19415bde0 0x7ff19406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff1989e7700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1940b4240 0x7ff19415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff1989e7700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1940cc670 con 0x7ff19415c1c0 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff1989e7700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940cffb0 con 0x7ff1940b4240 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff1989e7700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1940cf7a0 con 0x7ff19415bde0 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff192d9d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1940b4240 0x7ff19415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff191d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff19415c1c0 0x7ff1940cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53836/0 (socket says 172.21.15.26:53836) 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff191d9b700 1 -- 172.21.15.26:0/655730140 learned_addr learned my addr 172.21.15.26:0/655730140 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:37.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff19259c700 1 --2- 172.21.15.26:0/655730140 >> v2:172.21.15.93:3300/0 conn(0x7ff19415bde0 0x7ff19406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff192d9d700 1 -- 172.21.15.26:0/655730140 >> v2:172.21.15.93:3300/0 conn(0x7ff19415bde0 msgr2=0x7ff19406ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff192d9d700 1 --2- 172.21.15.26:0/655730140 >> v2:172.21.15.93:3300/0 conn(0x7ff19415bde0 0x7ff19406ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff192d9d700 1 -- 172.21.15.26:0/655730140 >> v1:172.21.15.26:6789/0 conn(0x7ff19415c1c0 legacy=0x7ff1940cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.711+0000 7ff192d9d700 1 -- 172.21.15.26:0/655730140 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940cf9f0 con 0x7ff1940b4240 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff192d9d700 1 --2- 172.21.15.26:0/655730140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1940b4240 0x7ff19415fab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff19406bf70 tx=0x7ff18c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ade0619c16223e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff19159a700 1 -- 172.21.15.26:0/655730140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00e070 con 0x7ff1940b4240 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff19159a700 1 -- 172.21.15.26:0/655730140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c00bbd0 con 0x7ff1940b4240 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff19159a700 1 -- 172.21.15.26:0/655730140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c012430 con 0x7ff1940b4240 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 -- 172.21.15.26:0/655730140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1940b4240 msgr2=0x7ff19415fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 --2- 172.21.15.26:0/655730140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1940b4240 0x7ff19415fab0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff19406bf70 tx=0x7ff18c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 -- 172.21.15.26:0/655730140 shutdown_connections 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 --2- 172.21.15.26:0/655730140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1940b4240 0x7ff19415fab0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 --2- 172.21.15.26:0/655730140 >> v2:172.21.15.93:3300/0 conn(0x7ff19415bde0 0x7ff19406ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 -- 172.21.15.26:0/655730140 >> 172.21.15.26:0/655730140 conn(0x7ff1940b3720 msgr2=0x7ff1940b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 -- 172.21.15.26:0/655730140 shutdown_connections 2024-01-16T14:55:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.712+0000 7ff1989e7700 1 -- 172.21.15.26:0/655730140 wait complete. 2024-01-16T14:55:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.714+0000 7ff1989e7700 1 Processor -- start 2024-01-16T14:55:37.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.714+0000 7ff1989e7700 1 -- start start 2024-01-16T14:55:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1940b4240 0x7ff1940b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff19415ed90 0x7ff1940b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194152860 con 0x7ff19415bde0 2024-01-16T14:55:37.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19415e010 con 0x7ff19415ed90 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.715+0000 7ff1989e7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940b57e0 con 0x7ff1940b4240 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36636/0 (socket says 172.21.15.26:36636) 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 -- 172.21.15.26:0/421945323 learned_addr learned my addr 172.21.15.26:0/421945323 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff192d9d700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff19415ed90 0x7ff1940b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff19259c700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1940b4240 0x7ff1940b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 -- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1940b4240 msgr2=0x7ff1940b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1940b4240 0x7ff1940b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 -- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff19415ed90 msgr2=0x7ff1940b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff19415ed90 0x7ff1940b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 -- 172.21.15.26:0/421945323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c009a70 con 0x7ff19415bde0 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff191d9b700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c480 tx=0x7ff18800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188005d70 con 0x7ff19415bde0 2024-01-16T14:55:37.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1880070d0 con 0x7ff19415bde0 2024-01-16T14:55:37.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19415e2b0 con 0x7ff19415bde0 2024-01-16T14:55:37.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188007440 con 0x7ff19415bde0 2024-01-16T14:55:37.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.716+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19415e870 con 0x7ff19415bde0 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.717+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff194075480 con 0x7ff19415bde0 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.718+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff1880075a0 con 0x7ff19415bde0 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.718+0000 7ff182ffd700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 0x7ff17406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.718+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff188088c60 con 0x7ff19415bde0 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.719+0000 7ff19259c700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 0x7ff17406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:37.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.719+0000 7ff19259c700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 0x7ff17406d810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff184008850 tx=0x7ff184008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:37.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:37.723+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff188053ec0 con 0x7ff19415bde0 2024-01-16T14:55:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.018+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff1940b7b50 con 0x7ff19415bde0 2024-01-16T14:55:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.020+0000 7ff182ffd700 1 -- 172.21.15.26:0/421945323 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff1880170a0 con 0x7ff19415bde0 2024-01-16T14:55:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 msgr2=0x7ff17406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 0x7ff17406d810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff184008850 tx=0x7ff184008040 comp rx=0 tx=0).stop 2024-01-16T14:55:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 msgr2=0x7ff194151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff18800c480 tx=0x7ff18800c840 comp rx=0 tx=0).stop 2024-01-16T14:55:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 shutdown_connections 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff19415ed90 0x7ff1940b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1940b4240 0x7ff1940b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff17406b350 0x7ff17406d810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 --2- 172.21.15.26:0/421945323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19415bde0 0x7ff194151cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 >> 172.21.15.26:0/421945323 conn(0x7ff1940b3720 msgr2=0x7ff1940707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 shutdown_connections 2024-01-16T14:55:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.022+0000 7ff1989e7700 1 -- 172.21.15.26:0/421945323 wait complete. 2024-01-16T14:55:38.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-16T14:55:38.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.175+0000 7f04efaea700 1 Processor -- start 2024-01-16T14:55:38.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.175+0000 7f04efaea700 1 -- start start 2024-01-16T14:55:38.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04efaea700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04e815e1c0 0x7f04e806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04efaea700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f04e806fc50 0x7f04e806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04efaea700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04e80d1230 con 0x7f04e80b0e90 2024-01-16T14:55:38.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04efaea700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e80d0c70 con 0x7f04e815e1c0 2024-01-16T14:55:38.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04efaea700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04e80d0f50 con 0x7f04e806fc50 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04e815e1c0 0x7f04e806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04e815e1c0 0x7f04e806eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36648/0 (socket says 172.21.15.26:36648) 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 -- 172.21.15.26:0/1691187702 learned_addr learned my addr 172.21.15.26:0/1691187702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 -- 172.21.15.26:0/1691187702 >> v2:172.21.15.93:3300/0 conn(0x7f04e806fc50 msgr2=0x7f04e806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 --2- 172.21.15.26:0/1691187702 >> v2:172.21.15.93:3300/0 conn(0x7f04e806fc50 0x7f04e806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 -- 172.21.15.26:0/1691187702 >> v1:172.21.15.93:6789/0 conn(0x7f04e80b0e90 legacy=0x7f04e806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 -- 172.21.15.26:0/1691187702 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e806c210 con 0x7f04e815e1c0 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.176+0000 7f04ed886700 1 --2- 172.21.15.26:0/1691187702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e815e1c0 0x7f04e806eef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04e8162e30 tx=0x7f04e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb0a6c1e48990d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04ec884700 1 -- 172.21.15.26:0/1691187702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4004230 con 0x7f04e815e1c0 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04ec884700 1 -- 172.21.15.26:0/1691187702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4004390 con 0x7f04e815e1c0 2024-01-16T14:55:38.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04ec884700 1 -- 172.21.15.26:0/1691187702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e40036a0 con 0x7f04e815e1c0 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 -- 172.21.15.26:0/1691187702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e815e1c0 msgr2=0x7f04e806eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 --2- 172.21.15.26:0/1691187702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e815e1c0 0x7f04e806eef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f04e8162e30 tx=0x7f04e4009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 -- 172.21.15.26:0/1691187702 shutdown_connections 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 --2- 172.21.15.26:0/1691187702 >> v2:172.21.15.93:3300/0 conn(0x7f04e806fc50 0x7f04e806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 --2- 172.21.15.26:0/1691187702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e815e1c0 0x7f04e806eef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 -- 172.21.15.26:0/1691187702 >> 172.21.15.26:0/1691187702 conn(0x7f04e80b3720 msgr2=0x7f04e80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:38.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 -- 172.21.15.26:0/1691187702 shutdown_connections 2024-01-16T14:55:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.177+0000 7f04efaea700 1 -- 172.21.15.26:0/1691187702 wait complete. 2024-01-16T14:55:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.179+0000 7f04efaea700 1 Processor -- start 2024-01-16T14:55:38.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.179+0000 7f04efaea700 1 -- start start 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04e806fc50 0x7f04e814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04e815e1c0 0x7f04e81f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e814f580 con 0x7f04e80b0e90 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e81f5dc0 con 0x7f04e815e1c0 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04efaea700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e81f6060 con 0x7f04e806fc50 2024-01-16T14:55:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04ed085700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04ed085700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36664/0 (socket says 172.21.15.26:36664) 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04ed085700 1 -- 172.21.15.26:0/1897330431 learned_addr learned my addr 172.21.15.26:0/1897330431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04ed886700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04e806fc50 0x7f04e814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.180+0000 7f04ee087700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04e815e1c0 0x7f04e81f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 -- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04e806fc50 msgr2=0x7f04e814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04e806fc50 0x7f04e814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 -- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04e815e1c0 msgr2=0x7f04e81f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04e815e1c0 0x7f04e81f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 -- 172.21.15.26:0/1897330431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e4009550 con 0x7f04e80b0e90 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04ed085700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c4a0 tx=0x7f04d800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d80054e0 con 0x7f04e80b0e90 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d8003680 con 0x7f04e80b0e90 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e81f6340 con 0x7f04e80b0e90 2024-01-16T14:55:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d8003a30 con 0x7f04e80b0e90 2024-01-16T14:55:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.181+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e81f6900 con 0x7f04e80b0e90 2024-01-16T14:55:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.182+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e8075480 con 0x7f04e80b0e90 2024-01-16T14:55:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.183+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f04d8003b90 con 0x7f04e80b0e90 2024-01-16T14:55:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.183+0000 7f04de7fc700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 0x7f04cc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.183+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04d8088820 con 0x7f04e80b0e90 2024-01-16T14:55:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.183+0000 7f04ed886700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 0x7f04cc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.184+0000 7f04ed886700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 0x7f04cc06d840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009130 tx=0x7f04e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.188+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04d8053950 con 0x7f04e80b0e90 2024-01-16T14:55:38.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.486+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f04e80b4240 con 0x7f04e80b0e90 2024-01-16T14:55:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.488+0000 7f04de7fc700 1 -- 172.21.15.26:0/1897330431 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f04d80170a0 con 0x7f04e80b0e90 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 msgr2=0x7f04cc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 0x7f04cc06d840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04e4009130 tx=0x7f04e4008030 comp rx=0 tx=0).stop 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 msgr2=0x7f04e815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f04d800c4a0 tx=0x7f04d800c7b0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 shutdown_connections 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04e815e1c0 0x7f04e81f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04e806fc50 0x7f04e814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f04cc06b380 0x7f04cc06d840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 --2- 172.21.15.26:0/1897330431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04e80b0e90 0x7f04e815a300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 >> 172.21.15.26:0/1897330431 conn(0x7f04e80b3720 msgr2=0x7f04e806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 shutdown_connections 2024-01-16T14:55:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.491+0000 7f04efaea700 1 -- 172.21.15.26:0/1897330431 wait complete. 2024-01-16T14:55:38.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-16T14:55:38.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.641+0000 7f902ef11700 1 Processor -- start 2024-01-16T14:55:38.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 -- start start 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f902806afd0 0x7f90280d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f90280adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9028070490 con 0x7f90280affb0 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90280b0410 con 0x7f902806afd0 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.642+0000 7f902ef11700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280af9f0 con 0x7f90280ada10 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f902806afd0 0x7f90280d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f9027fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90280affb0 0x7f902806b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53840/0 (socket says 172.21.15.26:53840) 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f9027fff700 1 -- 172.21.15.26:0/3055423879 learned_addr learned my addr 172.21.15.26:0/3055423879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902d4ae700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f90280adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 -- 172.21.15.26:0/3055423879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 msgr2=0x7f90280adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f90280adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 -- 172.21.15.26:0/3055423879 >> v1:172.21.15.26:6789/0 conn(0x7f90280affb0 legacy=0x7f902806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 -- 172.21.15.26:0/3055423879 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90280afcd0 con 0x7f902806afd0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f90277fe700 1 -- 172.21.15.26:0/3055423879 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515458984 0 0) 0x7f9028070490 con 0x7f90280affb0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f902ccad700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f902806afd0 0x7f90280d16e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f901c009840 tx=0x7f901c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95c9bdff88d875 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f90277fe700 1 -- 172.21.15.26:0/3055423879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c011830 con 0x7f902806afd0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f90277fe700 1 -- 172.21.15.26:0/3055423879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c011990 con 0x7f902806afd0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.643+0000 7f90277fe700 1 -- 172.21.15.26:0/3055423879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c011d20 con 0x7f902806afd0 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 -- 172.21.15.26:0/3055423879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f902806afd0 msgr2=0x7f90280d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f902806afd0 0x7f90280d16e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f901c009840 tx=0x7f901c009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 -- 172.21.15.26:0/3055423879 shutdown_connections 2024-01-16T14:55:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f90280adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 --2- 172.21.15.26:0/3055423879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f902806afd0 0x7f90280d16e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 -- 172.21.15.26:0/3055423879 >> 172.21.15.26:0/3055423879 conn(0x7f90280d0bc0 msgr2=0x7f90280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 -- 172.21.15.26:0/3055423879 shutdown_connections 2024-01-16T14:55:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.644+0000 7f902ef11700 1 -- 172.21.15.26:0/3055423879 wait complete. 2024-01-16T14:55:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.645+0000 7f902ef11700 1 Processor -- start 2024-01-16T14:55:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.645+0000 7f902ef11700 1 -- start start 2024-01-16T14:55:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f902806afd0 0x7f9028157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f9028152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028156cf0 con 0x7f90280affb0 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9028156a10 con 0x7f902806afd0 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902ef11700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90280cd700 con 0x7f90280ada10 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902d4ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902d4ae700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36684/0 (socket says 172.21.15.26:36684) 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.646+0000 7f902d4ae700 1 -- 172.21.15.26:0/3025017077 learned_addr learned my addr 172.21.15.26:0/3025017077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f9027fff700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f9028152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902ccad700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f902806afd0 0x7f9028157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 -- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 msgr2=0x7f9028152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f9028152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 -- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f902806afd0 msgr2=0x7f9028157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f902806afd0 0x7f9028157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 -- 172.21.15.26:0/3025017077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c0094a0 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902d4ae700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9020007e40 tx=0x7f902000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902000eea0 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9020014420 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902815f470 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90200147b0 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.647+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90280cdcc0 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.648+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9028075480 con 0x7f90280affb0 2024-01-16T14:55:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.649+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9020014910 con 0x7f90280affb0 2024-01-16T14:55:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.649+0000 7f90257fa700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 0x7f900c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.649+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9020089fa0 con 0x7f90280affb0 2024-01-16T14:55:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.649+0000 7f902ccad700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 0x7f900c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.650+0000 7f902ccad700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 0x7f900c06d810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f901c0090b0 tx=0x7f901c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.654+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90200bce90 con 0x7f90280affb0 2024-01-16T14:55:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.948+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f902806b730 con 0x7f90280affb0 2024-01-16T14:55:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.950+0000 7f90257fa700 1 -- 172.21.15.26:0/3025017077 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90200551f0 con 0x7f90280affb0 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 msgr2=0x7f900c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 0x7f900c06d810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f901c0090b0 tx=0x7f901c01c040 comp rx=0 tx=0).stop 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 msgr2=0x7f90280cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9020007e40 tx=0x7f902000e3c0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 shutdown_connections 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f900c06b350 0x7f900c06d810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90280affb0 0x7f90280cf2e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f902806afd0 0x7f9028157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 --2- 172.21.15.26:0/3025017077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f90280ada10 0x7f9028152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 >> 172.21.15.26:0/3025017077 conn(0x7f90280d0bc0 msgr2=0x7f90280b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 shutdown_connections 2024-01-16T14:55:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:38.953+0000 7f902ef11700 1 -- 172.21.15.26:0/3025017077 wait complete. 2024-01-16T14:55:38.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-16T14:55:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.106+0000 7f5112425700 1 Processor -- start 2024-01-16T14:55:39.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.106+0000 7f5112425700 1 -- start start 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.107+0000 7f5112425700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.107+0000 7f5112425700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 0x7f510c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.107+0000 7f5112425700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c0b1d20 con 0x7f510c0b5e40 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.107+0000 7f5112425700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c06bb80 con 0x7f510c06b400 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.107+0000 7f5112425700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c06be20 con 0x7f510c15e1e0 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f51109c2700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f510c0b5e40 0x7f510c0726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53852/0 (socket says 172.21.15.26:53852) 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f51109c2700 1 -- 172.21.15.26:0/73594858 learned_addr learned my addr 172.21.15.26:0/73594858 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 --2- 172.21.15.26:0/73594858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510b7fe700 1 --2- 172.21.15.26:0/73594858 >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 0x7f510c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678682723 0 0) 0x7f510c0b1d20 con 0x7f510c0b5e40 2024-01-16T14:55:39.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f40035b0 con 0x7f510c0b5e40 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 -- 172.21.15.26:0/73594858 >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 msgr2=0x7f510c0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 --2- 172.21.15.26:0/73594858 >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 0x7f510c0b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 -- 172.21.15.26:0/73594858 >> v1:172.21.15.26:6789/0 conn(0x7f510c0b5e40 legacy=0x7f510c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096348296 0 0) 0x7f50f40035b0 con 0x7f510c0b5e40 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 -- 172.21.15.26:0/73594858 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c06c100 con 0x7f510c15e1e0 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510b7fe700 1 --2- 172.21.15.26:0/73594858 >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 0x7f510c0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.108+0000 7f510bfff700 1 --2- 172.21.15.26:0/73594858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c0b12a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009840 tx=0x7f50fc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237a251e424dbc5c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51040019f0 con 0x7f510c15e1e0 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc004030 con 0x7f510c15e1e0 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f510affd700 1 -- 172.21.15.26:0/73594858 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0043c0 con 0x7f510c15e1e0 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 -- 172.21.15.26:0/73594858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 msgr2=0x7f510c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 --2- 172.21.15.26:0/73594858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c0b12a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009840 tx=0x7f50fc009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 -- 172.21.15.26:0/73594858 shutdown_connections 2024-01-16T14:55:39.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 --2- 172.21.15.26:0/73594858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c0b12a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 --2- 172.21.15.26:0/73594858 >> v2:172.21.15.93:3300/0 conn(0x7f510c06b400 0x7f510c0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 -- 172.21.15.26:0/73594858 >> 172.21.15.26:0/73594858 conn(0x7f510c0b5070 msgr2=0x7f510c0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 -- 172.21.15.26:0/73594858 shutdown_connections 2024-01-16T14:55:39.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.109+0000 7f5112425700 1 -- 172.21.15.26:0/73594858 wait complete. 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.110+0000 7f5112425700 1 Processor -- start 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.111+0000 7f5112425700 1 -- start start 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.111+0000 7f5112425700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 0x7f510c1e7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c14c000 con 0x7f510c06b400 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c09e370 con 0x7f510c0b5e40 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c1e7dd0 con 0x7f510c15e1e0 2024-01-16T14:55:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36698/0 (socket says 172.21.15.26:36698) 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 -- 172.21.15.26:0/3463807605 learned_addr learned my addr 172.21.15.26:0/3463807605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f51109c2700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510b7fe700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 0x7f510c1e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 -- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 msgr2=0x7f510c14bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c14bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 -- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 msgr2=0x7f510c1e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 0x7f510c1e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 -- 172.21.15.26:0/3463807605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000094a0 con 0x7f510c06b400 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510b7fe700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 0x7f510c1e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f510bfff700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00a010 tx=0x7f50fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003c40 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc01f070 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc011630 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.112+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc0094a0 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.113+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c1e8390 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.113+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c075480 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.114+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f50fc021030 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.114+0000 7f5108ff9700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 0x7f50f806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.114+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f50fc08abe0 con 0x7f510c06b400 2024-01-16T14:55:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.115+0000 7f510b7fe700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 0x7f50f806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.115+0000 7f510b7fe700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 0x7f50f806d810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f510c0d0fe0 tx=0x7f5100008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.119+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50fc055d90 con 0x7f510c06b400 2024-01-16T14:55:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.418+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f510c15dc50 con 0x7f510c06b400 2024-01-16T14:55:39.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.420+0000 7f5108ff9700 1 -- 172.21.15.26:0/3463807605 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f50fc0593b0 con 0x7f510c06b400 2024-01-16T14:55:39.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 msgr2=0x7f50f806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 0x7f50f806d810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f510c0d0fe0 tx=0x7f5100008040 comp rx=0 tx=0).stop 2024-01-16T14:55:39.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 msgr2=0x7f510c15c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00a010 tx=0x7f50fc004750 comp rx=0 tx=0).stop 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 shutdown_connections 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f510c15e1e0 0x7f510c14bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f50f806b350 0x7f50f806d810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f510c06b400 0x7f510c15c0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 --2- 172.21.15.26:0/3463807605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f510c0b5e40 0x7f510c1e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 >> 172.21.15.26:0/3463807605 conn(0x7f510c0b5070 msgr2=0x7f510c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 shutdown_connections 2024-01-16T14:55:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.422+0000 7f5112425700 1 -- 172.21.15.26:0/3463807605 wait complete. 2024-01-16T14:55:39.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.572+0000 7fe7585ad700 1 Processor -- start 2024-01-16T14:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.572+0000 7fe7585ad700 1 -- start start 2024-01-16T14:55:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe7585ad700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe7585ad700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe7585ad700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750143ec0 con 0x7fe7500720f0 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe7585ad700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7500b40a0 con 0x7fe7501440f0 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe7585ad700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe750072580 con 0x7fe750143ae0 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe755b48700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:56132/0 (socket says 172.21.15.26:56132) 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756b4a700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fe7500720f0 0x7fe75015e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:38446/0 (socket says 172.21.15.26:38446) 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 -- 172.21.15.26:0/907281821 learned_addr learned my addr 172.21.15.26:0/907281821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe755b48700 1 --2- 172.21.15.26:0/907281821 >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:44666/0 (socket says 172.21.15.26:44666) 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 -- 172.21.15.26:0/907281821 >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 msgr2=0x7fe750071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 --2- 172.21.15.26:0/907281821 >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 -- 172.21.15.26:0/907281821 >> v1:172.21.15.93:6789/0 conn(0x7fe7500720f0 legacy=0x7fe75015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe756349700 1 -- 172.21.15.26:0/907281821 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500b3ae0 con 0x7fe750143ae0 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe755347700 1 -- 172.21.15.26:0/907281821 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000165986 0 0) 0x7fe750143ec0 con 0x7fe7500720f0 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.574+0000 7fe755b48700 1 --2- 172.21.15.26:0/907281821 >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe756349700 1 --2- 172.21.15.26:0/907281821 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098f0 tx=0x7fe744009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee3d78ea52cfb46 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe755347700 1 -- 172.21.15.26:0/907281821 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744011830 con 0x7fe750143ae0 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe755347700 1 -- 172.21.15.26:0/907281821 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744011990 con 0x7fe750143ae0 2024-01-16T14:55:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe755347700 1 -- 172.21.15.26:0/907281821 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744011d20 con 0x7fe750143ae0 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe7585ad700 1 -- 172.21.15.26:0/907281821 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 msgr2=0x7fe7500b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.575+0000 7fe7585ad700 1 --2- 172.21.15.26:0/907281821 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7440098f0 tx=0x7fe744009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 -- 172.21.15.26:0/907281821 shutdown_connections 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 --2- 172.21.15.26:0/907281821 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe750143ae0 0x7fe7500b51b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 --2- 172.21.15.26:0/907281821 >> v2:172.21.15.93:3300/0 conn(0x7fe7501440f0 0x7fe750071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 -- 172.21.15.26:0/907281821 >> 172.21.15.26:0/907281821 conn(0x7fe7500b2740 msgr2=0x7fe7500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 -- 172.21.15.26:0/907281821 shutdown_connections 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.576+0000 7fe7585ad700 1 -- 172.21.15.26:0/907281821 wait complete. 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.577+0000 7fe7585ad700 1 Processor -- start 2024-01-16T14:55:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.577+0000 7fe7585ad700 1 -- start start 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7500720f0 0x7fe7501efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7501440f0 0x7fe750134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750153090 con 0x7fe750143ae0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750153330 con 0x7fe7500720f0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe7585ad700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe750153610 con 0x7fe7501440f0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36702/0 (socket says 172.21.15.26:36702) 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 -- 172.21.15.26:0/2362075154 learned_addr learned my addr 172.21.15.26:0/2362075154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe756b4a700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7501440f0 0x7fe750134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe756349700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7500720f0 0x7fe7501efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 -- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7501440f0 msgr2=0x7fe750134e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7501440f0 0x7fe750134e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 -- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7500720f0 msgr2=0x7fe7501efae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7500720f0 0x7fe7501efae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.578+0000 7fe755b48700 1 -- 172.21.15.26:0/2362075154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0094a0 con 0x7fe750143ae0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe755b48700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00ebe0 tx=0x7fe74c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00c850 con 0x7fe750143ae0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00c9b0 con 0x7fe750143ae0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe744009550 con 0x7fe750143ae0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00cd40 con 0x7fe750143ae0 2024-01-16T14:55:39.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.579+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe750153bd0 con 0x7fe750143ae0 2024-01-16T14:55:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.580+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750154220 con 0x7fe750143ae0 2024-01-16T14:55:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.580+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe74c013470 con 0x7fe750143ae0 2024-01-16T14:55:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.581+0000 7fe742ffd700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 0x7fe73406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.581+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe74c0898c0 con 0x7fe750143ae0 2024-01-16T14:55:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.581+0000 7fe756349700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 0x7fe73406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.581+0000 7fe756349700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 0x7fe73406d810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.586+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe74c054a80 con 0x7fe750143ae0 2024-01-16T14:55:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.799+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe750075480 con 0x7fe750143ae0 2024-01-16T14:55:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.800+0000 7fe742ffd700 1 -- 172.21.15.26:0/2362075154 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12797 (secure 0 0 0) 0x7fe74c00cea0 con 0x7fe750143ae0 2024-01-16T14:55:39.783 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:39.783 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:55:36.156156+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:39.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 msgr2=0x7fe73406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 0x7fe73406d810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe744000c00 tx=0x7fe744008030 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 msgr2=0x7fe750152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00ebe0 tx=0x7fe74c00eef0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 shutdown_connections 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7501440f0 0x7fe750134e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7500720f0 0x7fe7501efae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe73406b350 0x7fe73406d810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 --2- 172.21.15.26:0/2362075154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe750143ae0 0x7fe750152b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.802+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 >> 172.21.15.26:0/2362075154 conn(0x7fe7500b2740 msgr2=0x7fe7500cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.803+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 shutdown_connections 2024-01-16T14:55:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.803+0000 7fe7585ad700 1 -- 172.21.15.26:0/2362075154 wait complete. 2024-01-16T14:55:39.799 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:55:39.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:55:39.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.957+0000 7fbb136d2700 1 Processor -- start 2024-01-16T14:55:39.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.957+0000 7fbb136d2700 1 -- start start 2024-01-16T14:55:39.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb136d2700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb0c0cecd0 0x7fbb0c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb136d2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c0d0a90 con 0x7fbb0c0cf2b0 2024-01-16T14:55:39.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb136d2700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c06ee10 con 0x7fbb0c15bde0 2024-01-16T14:55:39.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb136d2700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0c06f0f0 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb10c6d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb0c0cf2b0 0x7fbb0c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53864/0 (socket says 172.21.15.26:53864) 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb10c6d700 1 -- 172.21.15.26:0/4134326122 learned_addr learned my addr 172.21.15.26:0/4134326122 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb11c6f700 1 --2- 172.21.15.26:0/4134326122 >> v2:172.21.15.93:3300/0 conn(0x7fbb0c0cecd0 0x7fbb0c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55730155 0 0) 0x7fbb0c0d0a90 con 0x7fbb0c0cf2b0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaf80035b0 con 0x7fbb0c0cf2b0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764037456 0 0) 0x7fbb0c06ee10 con 0x7fbb0c15bde0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c0d0a90 con 0x7fbb0c15bde0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb11c6f700 1 -- 172.21.15.26:0/4134326122 >> v1:172.21.15.93:6789/0 conn(0x7fbb0c15bde0 legacy=0x7fbb0c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb11c6f700 1 -- 172.21.15.26:0/4134326122 >> v1:172.21.15.26:6789/0 conn(0x7fbb0c0cf2b0 legacy=0x7fbb0c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.958+0000 7fbb11c6f700 1 -- 172.21.15.26:0/4134326122 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c06f3d0 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb11c6f700 1 --2- 172.21.15.26:0/4134326122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c15c1c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009d90 tx=0x7fbb0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5156fb9f05ec0a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400e070 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0400bb20 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb03fff700 1 -- 172.21.15.26:0/4134326122 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04012430 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb136d2700 1 -- 172.21.15.26:0/4134326122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 msgr2=0x7fbb0c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb136d2700 1 --2- 172.21.15.26:0/4134326122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c15c1c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009d90 tx=0x7fbb0400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb136d2700 1 -- 172.21.15.26:0/4134326122 shutdown_connections 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb136d2700 1 --2- 172.21.15.26:0/4134326122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c15c1c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.959+0000 7fbb136d2700 1 -- 172.21.15.26:0/4134326122 >> 172.21.15.26:0/4134326122 conn(0x7fbb0c0b3720 msgr2=0x7fbb0c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.960+0000 7fbb136d2700 1 -- 172.21.15.26:0/4134326122 shutdown_connections 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.960+0000 7fbb136d2700 1 -- 172.21.15.26:0/4134326122 wait complete. 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.961+0000 7fbb136d2700 1 Processor -- start 2024-01-16T14:55:39.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.961+0000 7fbb136d2700 1 -- start start 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb0c0cf2b0 0x7fbb0c164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c06dd10 con 0x7fbb0c15bde0 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c165800 con 0x7fbb0c0cecd0 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb136d2700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c165ae0 con 0x7fbb0c0cf2b0 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36706/0 (socket says 172.21.15.26:36706) 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 -- 172.21.15.26:0/298655743 learned_addr learned my addr 172.21.15.26:0/298655743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb10c6d700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb0c0cf2b0 0x7fbb0c164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb1146e700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 -- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb0c0cf2b0 msgr2=0x7fbb0c164980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb0c0cf2b0 0x7fbb0c164980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 -- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 msgr2=0x7fbb0c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 -- 172.21.15.26:0/298655743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04009a70 con 0x7fbb0c15bde0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb1146e700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.962+0000 7fbb11c6f700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbb04013000 tx=0x7fbb04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.963+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0401b810 con 0x7fbb0c15bde0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.963+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb04020070 con 0x7fbb0c15bde0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.963+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c165dc0 con 0x7fbb0c15bde0 2024-01-16T14:55:39.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.963+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0401bbb0 con 0x7fbb0c15bde0 2024-01-16T14:55:39.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.963+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c166380 con 0x7fbb0c15bde0 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.964+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c075480 con 0x7fbb0c15bde0 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.964+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbb0401bd10 con 0x7fbb0c15bde0 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.965+0000 7fbb01ffb700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 0x7fbaf006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.965+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb04022070 con 0x7fbb0c15bde0 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.965+0000 7fbb1146e700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 0x7fbaf006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.966+0000 7fbb1146e700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 0x7fbaf006d810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb080011e0 tx=0x7fbb08009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:39.970+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb040568d0 con 0x7fbb0c15bde0 2024-01-16T14:55:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.180+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb0c0d2650 con 0x7fbb0c15bde0 2024-01-16T14:55:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.181+0000 7fbb01ffb700 1 -- 172.21.15.26:0/298655743 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb0402c020 con 0x7fbb0c15bde0 2024-01-16T14:55:40.163 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 msgr2=0x7fbaf006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 0x7fbaf006d810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbb080011e0 tx=0x7fbb08009040 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 msgr2=0x7fbb0c164ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbb04013000 tx=0x7fbb04004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 shutdown_connections 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fbaf006b350 0x7fbaf006d810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb0c15bde0 0x7fbb0c164ec0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb0c0cecd0 0x7fbb0c0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 --2- 172.21.15.26:0/298655743 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb0c0cf2b0 0x7fbb0c164980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 >> 172.21.15.26:0/298655743 conn(0x7fbb0c0b3720 msgr2=0x7fbb0c157470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 shutdown_connections 2024-01-16T14:55:40.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.183+0000 7fbb136d2700 1 -- 172.21.15.26:0/298655743 wait complete. 2024-01-16T14:55:40.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-16T14:55:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.335+0000 7fe17a7ce700 1 Processor -- start 2024-01-16T14:55:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.335+0000 7fe17a7ce700 1 -- start start 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.336+0000 7fe17a7ce700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 0x7fe1740b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.336+0000 7fe17a7ce700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe17406f420 0x7fe1740ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe17a7ce700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17406ef90 con 0x7fe1740b4c40 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe17a7ce700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1740b5560 con 0x7fe17406f420 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe17a7ce700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17406e380 con 0x7fe1740b3b40 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe173fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe1740b4c40 0x7fe17406dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53868/0 (socket says 172.21.15.26:53868) 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe173fff700 1 -- 172.21.15.26:0/4172414258 learned_addr learned my addr 172.21.15.26:0/4172414258 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe1737fe700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 0x7fe1740b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe178d6b700 1 --2- 172.21.15.26:0/4172414258 >> v2:172.21.15.93:3300/0 conn(0x7fe17406f420 0x7fe1740ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953367536 0 0) 0x7fe17406ef90 con 0x7fe1740b4c40 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe15c0057c0 con 0x7fe1740b4c40 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe178d6b700 1 -- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 msgr2=0x7fe1740b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe178d6b700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 0x7fe1740b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe178d6b700 1 -- 172.21.15.26:0/4172414258 >> v1:172.21.15.26:6789/0 conn(0x7fe1740b4c40 legacy=0x7fe17406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420973209 0 0) 0x7fe15c0057c0 con 0x7fe1740b4c40 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe178d6b700 1 -- 172.21.15.26:0/4172414258 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17406e660 con 0x7fe17406f420 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.337+0000 7fe1737fe700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 0x7fe1740b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe178d6b700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe17406f420 0x7fe1740ce1d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009d90 tx=0x7fe16c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0adb9b39841de84 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00e070 con 0x7fe17406f420 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c003a20 con 0x7fe17406f420 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe172ffd700 1 -- 172.21.15.26:0/4172414258 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164001670 con 0x7fe17406f420 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 -- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe17406f420 msgr2=0x7fe1740ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe17406f420 0x7fe1740ce1d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009d90 tx=0x7fe16c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 -- 172.21.15.26:0/4172414258 shutdown_connections 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe17406f420 0x7fe1740ce1d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/4172414258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b3b40 0x7fe1740b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 -- 172.21.15.26:0/4172414258 >> 172.21.15.26:0/4172414258 conn(0x7fe1740b2840 msgr2=0x7fe1740b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 -- 172.21.15.26:0/4172414258 shutdown_connections 2024-01-16T14:55:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.338+0000 7fe17a7ce700 1 -- 172.21.15.26:0/4172414258 wait complete. 2024-01-16T14:55:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.340+0000 7fe17a7ce700 1 Processor -- start 2024-01-16T14:55:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.340+0000 7fe17a7ce700 1 -- start start 2024-01-16T14:55:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 0x7fe174159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b4c40 0x7fe1741efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe174142970 con 0x7fe17406f420 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17415a1a0 con 0x7fe1740b3b40 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe17a7ce700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1740a9830 con 0x7fe1740b4c40 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe173fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe173fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36710/0 (socket says 172.21.15.26:36710) 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.341+0000 7fe173fff700 1 -- 172.21.15.26:0/3179187473 learned_addr learned my addr 172.21.15.26:0/3179187473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe178d6b700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b4c40 0x7fe1741efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe1737fe700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 0x7fe174159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 -- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b4c40 msgr2=0x7fe1741efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b4c40 0x7fe1741efa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 -- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 msgr2=0x7fe174159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 0x7fe174159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 -- 172.21.15.26:0/3179187473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680094a0 con 0x7fe17406f420 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe1737fe700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 0x7fe174159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe173fff700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c750 tx=0x7fe16400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164004c80 con 0x7fe17406f420 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe164004e00 con 0x7fe17406f420 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c009a70 con 0x7fe17406f420 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164007310 con 0x7fe17406f420 2024-01-16T14:55:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.342+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe174142600 con 0x7fe17406f420 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.343+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe174067c70 con 0x7fe17406f420 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.344+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe164007470 con 0x7fe17406f420 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.344+0000 7fe170ff9700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 0x7fe16006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.345+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe1640895f0 con 0x7fe17406f420 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.345+0000 7fe1737fe700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 0x7fe16006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.345+0000 7fe1737fe700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 0x7fe16006d810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b1c70 tx=0x7fe168008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.349+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1640547a0 con 0x7fe17406f420 2024-01-16T14:55:40.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.563+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe174075480 con 0x7fe17406f420 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.563+0000 7fe170ff9700 1 -- 172.21.15.26:0/3179187473 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe164057dc0 con 0x7fe17406f420 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:40.546 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:55:40.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 msgr2=0x7fe16006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 0x7fe16006d810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1740b1c70 tx=0x7fe168008040 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 msgr2=0x7fe174143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c750 tx=0x7fe16400cb10 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 shutdown_connections 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1740b4c40 0x7fe1741efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe16006b350 0x7fe16006d810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17406f420 0x7fe174143da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 --2- 172.21.15.26:0/3179187473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1740b3b40 0x7fe174159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.565+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 >> 172.21.15.26:0/3179187473 conn(0x7fe1740b2840 msgr2=0x7fe17415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.566+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 shutdown_connections 2024-01-16T14:55:40.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.566+0000 7fe17a7ce700 1 -- 172.21.15.26:0/3179187473 wait complete. 2024-01-16T14:55:40.562 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-16T14:55:40.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-16T14:55:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.719+0000 7ff1dfe2d700 1 Processor -- start 2024-01-16T14:55:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.720+0000 7ff1dfe2d700 1 -- start start 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.720+0000 7ff1dfe2d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff1d806bfe0 0x7ff1d806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.720+0000 7ff1dfe2d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806d2a0 con 0x7ff1d80722e0 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.720+0000 7ff1dfe2d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806d540 con 0x7ff1d815bc10 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.720+0000 7ff1dfe2d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d80d0210 con 0x7ff1d806bfe0 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1dd3c8700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1d80722e0 0x7ff1d80cfb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53876/0 (socket says 172.21.15.26:53876) 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1dd3c8700 1 -- 172.21.15.26:0/3097643533 learned_addr learned my addr 172.21.15.26:0/3097643533 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1de3ca700 1 --2- 172.21.15.26:0/3097643533 >> v2:172.21.15.93:3300/0 conn(0x7ff1d806bfe0 0x7ff1d806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1dcbc7700 1 -- 172.21.15.26:0/3097643533 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675594767 0 0) 0x7ff1d806d2a0 con 0x7ff1d80722e0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1dcbc7700 1 -- 172.21.15.26:0/3097643533 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c40035b0 con 0x7ff1d80722e0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1de3ca700 1 -- 172.21.15.26:0/3097643533 >> v1:172.21.15.93:6789/0 conn(0x7ff1d815bc10 legacy=0x7ff1d815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1de3ca700 1 -- 172.21.15.26:0/3097643533 >> v1:172.21.15.26:6789/0 conn(0x7ff1d80722e0 legacy=0x7ff1d80cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1de3ca700 1 -- 172.21.15.26:0/3097643533 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d806d820 con 0x7ff1d806bfe0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.721+0000 7ff1de3ca700 1 --2- 172.21.15.26:0/3097643533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d806c470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009d90 tx=0x7ff1d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d22b327edb60392 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dcbc7700 1 -- 172.21.15.26:0/3097643533 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e070 con 0x7ff1d806bfe0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dcbc7700 1 -- 172.21.15.26:0/3097643533 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000bb20 con 0x7ff1d806bfe0 2024-01-16T14:55:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dcbc7700 1 -- 172.21.15.26:0/3097643533 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0012430 con 0x7ff1d806bfe0 2024-01-16T14:55:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/3097643533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 msgr2=0x7ff1d806c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/3097643533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d806c470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009d90 tx=0x7ff1d000b330 comp rx=0 tx=0).stop 2024-01-16T14:55:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/3097643533 shutdown_connections 2024-01-16T14:55:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/3097643533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d806c470 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/3097643533 >> 172.21.15.26:0/3097643533 conn(0x7ff1d80b2830 msgr2=0x7ff1d8071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/3097643533 shutdown_connections 2024-01-16T14:55:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.722+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/3097643533 wait complete. 2024-01-16T14:55:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 Processor -- start 2024-01-16T14:55:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 -- start start 2024-01-16T14:55:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d8143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d80722e0 0x7ff1d81476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8144180 con 0x7ff1d815bc10 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8147be0 con 0x7ff1d806bfe0 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1dfe2d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d81552e0 con 0x7ff1d80722e0 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.724+0000 7ff1de3ca700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36716/0 (socket says 172.21.15.26:36716) 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 -- 172.21.15.26:0/1140335453 learned_addr learned my addr 172.21.15.26:0/1140335453 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1dd3c8700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d80722e0 0x7ff1d81476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1ddbc9700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d8143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 -- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d80722e0 msgr2=0x7ff1d81476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d80722e0 0x7ff1d81476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 -- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 msgr2=0x7ff1d8143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d8143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 -- 172.21.15.26:0/1140335453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009a70 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1de3ca700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80b5900 tx=0x7ff1d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0012430 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0020070 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0004280 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d81555c0 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.725+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d8146ea0 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.726+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8068670 con 0x7ff1d815bc10 2024-01-16T14:55:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.727+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff1d00043e0 con 0x7ff1d815bc10 2024-01-16T14:55:40.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.727+0000 7ff1ca7fc700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 0x7ff1bc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:40.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.728+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff1d0022070 con 0x7ff1d815bc10 2024-01-16T14:55:40.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.728+0000 7ff1ddbc9700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 0x7ff1bc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:40.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.728+0000 7ff1ddbc9700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 0x7ff1bc06d810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d814e1f0 tx=0x7ff1cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:40.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.732+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1d0056790 con 0x7ff1d815bc10 2024-01-16T14:55:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:40.943+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff1d8160820 con 0x7ff1d815bc10 2024-01-16T14:55:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.161+0000 7ff1ca7fc700 1 -- 172.21.15.26:0/1140335453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff1d0059db0 con 0x7ff1d815bc10 2024-01-16T14:55:41.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.165+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 msgr2=0x7ff1bc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.165+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 0x7ff1bc06d810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1d814e1f0 tx=0x7ff1cc009040 comp rx=0 tx=0).stop 2024-01-16T14:55:41.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.165+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 msgr2=0x7ff1d815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.165+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80b5900 tx=0x7ff1d0004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 shutdown_connections 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff1bc06b350 0x7ff1bc06d810 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d815bc10 0x7ff1d815acd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d806bfe0 0x7ff1d8143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 --2- 172.21.15.26:0/1140335453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d80722e0 0x7ff1d81476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 >> 172.21.15.26:0/1140335453 conn(0x7ff1d80b2830 msgr2=0x7ff1d814e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 shutdown_connections 2024-01-16T14:55:41.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.167+0000 7ff1dfe2d700 1 -- 172.21.15.26:0/1140335453 wait complete. 2024-01-16T14:55:41.154 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-16T14:55:41.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-16T14:55:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.332+0000 7f38d8b61700 1 Processor -- start 2024-01-16T14:55:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.332+0000 7f38d8b61700 1 -- start start 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.333+0000 7f38d8b61700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38d4143ae0 0x7f38d40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.333+0000 7f38d8b61700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f38d41440f0 0x7f38d4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.333+0000 7f38d8b61700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d4143ec0 con 0x7f38d40720f0 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.333+0000 7f38d8b61700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40b40a0 con 0x7f38d4143ae0 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.333+0000 7f38d8b61700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d4072560 con 0x7f38d41440f0 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38d4143ae0 0x7f38d40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d2d9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38d40720f0 0x7f38d415e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53882/0 (socket says 172.21.15.26:53882) 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38d4143ae0 0x7f38d40b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36728/0 (socket says 172.21.15.26:36728) 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d2d9d700 1 -- 172.21.15.26:0/3732669202 learned_addr learned my addr 172.21.15.26:0/3732669202 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d1d9b700 1 --2- 172.21.15.26:0/3732669202 >> v2:172.21.15.93:3300/0 conn(0x7f38d41440f0 0x7f38d4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 -- 172.21.15.26:0/3732669202 >> v2:172.21.15.93:3300/0 conn(0x7f38d41440f0 msgr2=0x7f38d4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 --2- 172.21.15.26:0/3732669202 >> v2:172.21.15.93:3300/0 conn(0x7f38d41440f0 0x7f38d4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 -- 172.21.15.26:0/3732669202 >> v1:172.21.15.26:6789/0 conn(0x7f38d40720f0 legacy=0x7f38d415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 -- 172.21.15.26:0/3732669202 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40b3ae0 con 0x7f38d4143ae0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d159a700 1 -- 172.21.15.26:0/3732669202 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263952552 0 0) 0x7f38d4143ec0 con 0x7f38d40720f0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d259c700 1 --2- 172.21.15.26:0/3732669202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d4143ae0 0x7f38d40b51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38d40d1b00 tx=0x7f38bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90194d2a9a07d7d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d159a700 1 -- 172.21.15.26:0/3732669202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011830 con 0x7f38d4143ae0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d159a700 1 -- 172.21.15.26:0/3732669202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc011990 con 0x7f38d4143ae0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.334+0000 7f38d159a700 1 -- 172.21.15.26:0/3732669202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011d20 con 0x7f38d4143ae0 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 -- 172.21.15.26:0/3732669202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d4143ae0 msgr2=0x7f38d40b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 --2- 172.21.15.26:0/3732669202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d4143ae0 0x7f38d40b51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f38d40d1b00 tx=0x7f38bc009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 -- 172.21.15.26:0/3732669202 shutdown_connections 2024-01-16T14:55:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 --2- 172.21.15.26:0/3732669202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d4143ae0 0x7f38d40b51b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 --2- 172.21.15.26:0/3732669202 >> v2:172.21.15.93:3300/0 conn(0x7f38d41440f0 0x7f38d4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 -- 172.21.15.26:0/3732669202 >> 172.21.15.26:0/3732669202 conn(0x7f38d40b2740 msgr2=0x7f38d40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:41.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 -- 172.21.15.26:0/3732669202 shutdown_connections 2024-01-16T14:55:41.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.335+0000 7f38d8b61700 1 -- 172.21.15.26:0/3732669202 wait complete. 2024-01-16T14:55:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 Processor -- start 2024-01-16T14:55:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 -- start start 2024-01-16T14:55:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d4143ae0 0x7f38d41535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d41440f0 0x7f38d4153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4142b20 con 0x7f38d40720f0 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4141f10 con 0x7f38d4143ae0 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d8b61700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41421f0 con 0x7f38d41440f0 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.337+0000 7f38d259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36730/0 (socket says 172.21.15.26:36730) 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 -- 172.21.15.26:0/1483068816 learned_addr learned my addr 172.21.15.26:0/1483068816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 -- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d41440f0 msgr2=0x7f38d4153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d41440f0 0x7f38d4153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 -- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d4143ae0 msgr2=0x7f38d41535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d4143ae0 0x7f38d41535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 -- 172.21.15.26:0/1483068816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0094a0 con 0x7f38d40720f0 2024-01-16T14:55:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d259c700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0038a0 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc01e070 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41424d0 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc003c20 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.338+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d413f5d0 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.339+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d4075480 con 0x7f38d40720f0 2024-01-16T14:55:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.340+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f38bc02d450 con 0x7f38d40720f0 2024-01-16T14:55:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.340+0000 7f38caffd700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 0x7f38b406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.340+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f38bc08a9a0 con 0x7f38d40720f0 2024-01-16T14:55:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.341+0000 7f38d1d9b700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 0x7f38b406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.341+0000 7f38d1d9b700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 0x7f38b406d810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098d0 tx=0x7f38c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.346+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38bc055a80 con 0x7f38d40720f0 2024-01-16T14:55:41.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:41.557+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f38d4144830 con 0x7f38d40720f0 2024-01-16T14:55:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.167+0000 7f38caffd700 1 -- 172.21.15.26:0/1483068816 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f38bc0150d0 con 0x7f38d40720f0 2024-01-16T14:55:42.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.170+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 msgr2=0x7f38b406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.170+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 0x7f38b406d810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098d0 tx=0x7f38c4008040 comp rx=0 tx=0).stop 2024-01-16T14:55:42.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.170+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 msgr2=0x7f38d413f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0090b0 tx=0x7f38bc004750 comp rx=0 tx=0).stop 2024-01-16T14:55:42.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 shutdown_connections 2024-01-16T14:55:42.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d41440f0 0x7f38d4153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f38b406b350 0x7f38b406d810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38d40720f0 0x7f38d413f090 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 --2- 172.21.15.26:0/1483068816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d4143ae0 0x7f38d41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.171+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 >> 172.21.15.26:0/1483068816 conn(0x7f38d40b2740 msgr2=0x7f38d40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:42.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.172+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 shutdown_connections 2024-01-16T14:55:42.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.172+0000 7f38d8b61700 1 -- 172.21.15.26:0/1483068816 wait complete. 2024-01-16T14:55:42.161 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-16T14:55:42.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-16T14:55:42.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.341+0000 7fdf64e14700 1 Processor -- start 2024-01-16T14:55:42.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.341+0000 7fdf64e14700 1 -- start start 2024-01-16T14:55:42.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf64e14700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf601457a0 0x7fdf600cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf64e14700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600d04b0 con 0x7fdf6006afd0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf64e14700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600cf1b0 con 0x7fdf600cd470 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf64e14700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600cf490 con 0x7fdf601457a0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf5e59c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf601457a0 0x7fdf600cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf5ed9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf6006afd0 0x7fdf6006b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53896/0 (socket says 172.21.15.26:53896) 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.342+0000 7fdf5ed9d700 1 -- 172.21.15.26:0/2967369513 learned_addr learned my addr 172.21.15.26:0/2967369513 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5e59c700 1 -- 172.21.15.26:0/2967369513 >> v1:172.21.15.93:6789/0 conn(0x7fdf600cd470 legacy=0x7fdf60160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5e59c700 1 -- 172.21.15.26:0/2967369513 >> v1:172.21.15.26:6789/0 conn(0x7fdf6006afd0 legacy=0x7fdf6006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5d59a700 1 -- 172.21.15.26:0/2967369513 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213200750 0 0) 0x7fdf600d04b0 con 0x7fdf6006afd0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5e59c700 1 -- 172.21.15.26:0/2967369513 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600cf770 con 0x7fdf601457a0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5e59c700 1 --2- 172.21.15.26:0/2967369513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600cff70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009840 tx=0x7fdf48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efa63e375812bd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5d59a700 1 -- 172.21.15.26:0/2967369513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011830 con 0x7fdf601457a0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5d59a700 1 -- 172.21.15.26:0/2967369513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48011990 con 0x7fdf601457a0 2024-01-16T14:55:42.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.343+0000 7fdf5d59a700 1 -- 172.21.15.26:0/2967369513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011d20 con 0x7fdf601457a0 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 -- 172.21.15.26:0/2967369513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 msgr2=0x7fdf600cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 --2- 172.21.15.26:0/2967369513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600cff70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009840 tx=0x7fdf48009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 -- 172.21.15.26:0/2967369513 shutdown_connections 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 --2- 172.21.15.26:0/2967369513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600cff70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 -- 172.21.15.26:0/2967369513 >> 172.21.15.26:0/2967369513 conn(0x7fdf600d0aa0 msgr2=0x7fdf600d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:42.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 -- 172.21.15.26:0/2967369513 shutdown_connections 2024-01-16T14:55:42.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.344+0000 7fdf64e14700 1 -- 172.21.15.26:0/2967369513 wait complete. 2024-01-16T14:55:42.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.345+0000 7fdf64e14700 1 Processor -- start 2024-01-16T14:55:42.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.345+0000 7fdf64e14700 1 -- start start 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf6006afd0 0x7fdf60152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 0x7fdf6015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60152fc0 con 0x7fdf601457a0 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600b3fd0 con 0x7fdf6006afd0 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf64e14700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600b42b0 con 0x7fdf600cd470 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf5ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf5ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36758/0 (socket says 172.21.15.26:36758) 2024-01-16T14:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.346+0000 7fdf5ed9d700 1 -- 172.21.15.26:0/1830512243 learned_addr learned my addr 172.21.15.26:0/1830512243 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5e59c700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf6006afd0 0x7fdf60152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5dd9b700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 0x7fdf6015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 -- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 msgr2=0x7fdf6015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 0x7fdf6015e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 -- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf6006afd0 msgr2=0x7fdf60152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf6006afd0 0x7fdf60152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 -- 172.21.15.26:0/1830512243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480094a0 con 0x7fdf601457a0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5dd9b700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 0x7fdf6015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf5ed9d700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf58002230 tx=0x7fdf5800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800ac20 con 0x7fdf601457a0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5800ada0 con 0x7fdf601457a0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58007310 con 0x7fdf601457a0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf600b4590 con 0x7fdf601457a0 2024-01-16T14:55:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.347+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf600b4b50 con 0x7fdf601457a0 2024-01-16T14:55:42.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.348+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600b4d70 con 0x7fdf601457a0 2024-01-16T14:55:42.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.349+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdf58007470 con 0x7fdf601457a0 2024-01-16T14:55:42.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.349+0000 7fdf56ffd700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 0x7fdf4006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:42.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.349+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdf58089650 con 0x7fdf601457a0 2024-01-16T14:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.349+0000 7fdf5e59c700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 0x7fdf4006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.350+0000 7fdf5e59c700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 0x7fdf4006d810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf48000c00 tx=0x7fdf4801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:42.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.354+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf58054510 con 0x7fdf601457a0 2024-01-16T14:55:42.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:42.617+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdf60075480 con 0x7fdf601457a0 2024-01-16T14:55:43.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.424+0000 7fdf56ffd700 1 -- 172.21.15.26:0/1830512243 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdf58057b30 con 0x7fdf601457a0 2024-01-16T14:55:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 msgr2=0x7fdf4006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 0x7fdf4006d810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf48000c00 tx=0x7fdf4801c040 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 msgr2=0x7fdf600b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf58002230 tx=0x7fdf5800a060 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 shutdown_connections 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fdf4006b350 0x7fdf4006d810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf601457a0 0x7fdf600b0540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdf6006afd0 0x7fdf60152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 --2- 172.21.15.26:0/1830512243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdf600cd470 0x7fdf6015e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 >> 172.21.15.26:0/1830512243 conn(0x7fdf600d0aa0 msgr2=0x7fdf600b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 shutdown_connections 2024-01-16T14:55:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.427+0000 7fdf64e14700 1 -- 172.21.15.26:0/1830512243 wait complete. 2024-01-16T14:55:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 4 and data pool 5 2024-01-16T14:55:43.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.587+0000 7f88af885700 1 Processor -- start 2024-01-16T14:55:43.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.588+0000 7f88af885700 1 -- start start 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88af885700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a806afd0 0x7f88a80b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88af885700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f88a8071b40 0x7f88a8071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88af885700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80ad860 con 0x7f88a806edc0 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88af885700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a806f830 con 0x7f88a806afd0 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88af885700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80d0a20 con 0x7f88a8071b40 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a806afd0 0x7f88a80b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a806afd0 0x7f88a80b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36762/0 (socket says 172.21.15.26:36762) 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ace20700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f88a806edc0 0x7f88a806b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53898/0 (socket says 172.21.15.26:53898) 2024-01-16T14:55:43.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 -- 172.21.15.26:0/286375832 learned_addr learned my addr 172.21.15.26:0/286375832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ade22700 1 --2- 172.21.15.26:0/286375832 >> v2:172.21.15.93:3300/0 conn(0x7f88a8071b40 0x7f88a8071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 -- 172.21.15.26:0/286375832 >> v2:172.21.15.93:3300/0 conn(0x7f88a8071b40 msgr2=0x7f88a8071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 --2- 172.21.15.26:0/286375832 >> v2:172.21.15.93:3300/0 conn(0x7f88a8071b40 0x7f88a8071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 -- 172.21.15.26:0/286375832 >> v1:172.21.15.26:6789/0 conn(0x7f88a806edc0 legacy=0x7f88a806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 -- 172.21.15.26:0/286375832 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a8162a70 con 0x7f88a806afd0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f889ffff700 1 -- 172.21.15.26:0/286375832 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098460827 0 0) 0x7f88a80ad860 con 0x7f88a806edc0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.589+0000 7f88ad621700 1 --2- 172.21.15.26:0/286375832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a80b5a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f88a80706d0 tx=0x7f88a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ad790c77197798 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f889ffff700 1 -- 172.21.15.26:0/286375832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4011830 con 0x7f88a806afd0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f889ffff700 1 -- 172.21.15.26:0/286375832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4011990 con 0x7f88a806afd0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f889ffff700 1 -- 172.21.15.26:0/286375832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4011d20 con 0x7f88a806afd0 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 -- 172.21.15.26:0/286375832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 msgr2=0x7f88a80b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 --2- 172.21.15.26:0/286375832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a80b5a70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f88a80706d0 tx=0x7f88a4009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 -- 172.21.15.26:0/286375832 shutdown_connections 2024-01-16T14:55:43.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 --2- 172.21.15.26:0/286375832 >> v2:172.21.15.93:3300/0 conn(0x7f88a8071b40 0x7f88a8071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 --2- 172.21.15.26:0/286375832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a80b5a70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 -- 172.21.15.26:0/286375832 >> 172.21.15.26:0/286375832 conn(0x7f88a80b2840 msgr2=0x7f88a80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 -- 172.21.15.26:0/286375832 shutdown_connections 2024-01-16T14:55:43.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.590+0000 7f88af885700 1 -- 172.21.15.26:0/286375832 wait complete. 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 Processor -- start 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 -- start start 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88a806edc0 0x7f88a815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88a8071b40 0x7f88a81604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.592+0000 7f88af885700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815f450 con 0x7f88a806afd0 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88af885700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815f730 con 0x7f88a806edc0 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88af885700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815fa10 con 0x7f88a8071b40 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36770/0 (socket says 172.21.15.26:36770) 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 -- 172.21.15.26:0/1112382472 learned_addr learned my addr 172.21.15.26:0/1112382472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ace20700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88a806edc0 0x7f88a815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ade22700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88a8071b40 0x7f88a81604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 -- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88a8071b40 msgr2=0x7f88a81604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88a8071b40 0x7f88a81604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 -- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88a806edc0 msgr2=0x7f88a815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88a806edc0 0x7f88a815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 -- 172.21.15.26:0/1112382472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009550 con 0x7f88a806afd0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88ad621700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009fd0 tx=0x7f88a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40036a0 con 0x7f88a806afd0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a401f070 con 0x7f88a806afd0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4003a20 con 0x7f88a806afd0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.593+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a815fcf0 con 0x7f88a806afd0 2024-01-16T14:55:43.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.594+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a81f7d40 con 0x7f88a806afd0 2024-01-16T14:55:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.595+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a81f7f90 con 0x7f88a806afd0 2024-01-16T14:55:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.595+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f88a4003b80 con 0x7f88a806afd0 2024-01-16T14:55:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.596+0000 7f889dffb700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 0x7f888c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.596+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a4021070 con 0x7f88a806afd0 2024-01-16T14:55:43.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.596+0000 7f88ace20700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 0x7f888c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.596+0000 7f88ace20700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 0x7f888c06d810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88a81546a0 tx=0x7f8898007be0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.601+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88a4056340 con 0x7f88a806afd0 2024-01-16T14:55:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.813+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88a8160210 con 0x7f88a806afd0 2024-01-16T14:55:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.814+0000 7f889dffb700 1 -- 172.21.15.26:0/1112382472 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f88a4059960 con 0x7f88a806afd0 2024-01-16T14:55:43.797 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":31,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:55:42.618310+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:55:40.975214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T14:55:41.589672+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 msgr2=0x7f888c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 0x7f888c06d810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f88a81546a0 tx=0x7f8898007be0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 msgr2=0x7f88a8142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009fd0 tx=0x7f88a4004750 comp rx=0 tx=0).stop 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 shutdown_connections 2024-01-16T14:55:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88a8071b40 0x7f88a81604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f888c06b350 0x7f888c06d810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806afd0 0x7f88a8142a30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 --2- 172.21.15.26:0/1112382472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88a806edc0 0x7f88a815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 >> 172.21.15.26:0/1112382472 conn(0x7f88a80b2840 msgr2=0x7f88a806e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 shutdown_connections 2024-01-16T14:55:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.817+0000 7f88af885700 1 -- 172.21.15.26:0/1112382472 wait complete. 2024-01-16T14:55:43.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.968+0000 7f401a81e700 1 Processor -- start 2024-01-16T14:55:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.968+0000 7f401a81e700 1 -- start start 2024-01-16T14:55:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.969+0000 7f401a81e700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f40140b3d90 0x7f401415faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.969+0000 7f401a81e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 0x7f401415ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.969+0000 7f401a81e700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401415f3e0 con 0x7f40140726a0 2024-01-16T14:55:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.969+0000 7f401a81e700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40140affd0 con 0x7f40140b3d90 2024-01-16T14:55:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.969+0000 7f401a81e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401415e7d0 con 0x7f4014073190 2024-01-16T14:55:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f40137fe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f40140b3d90 0x7f401415faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 0x7f401415ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4013fff700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f40140726a0 0x7f4014072a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:38464/0 (socket says 172.21.15.26:38464) 2024-01-16T14:55:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4013fff700 1 -- 172.21.15.26:0/1945373661 learned_addr learned my addr 172.21.15.26:0/1945373661 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 -- 172.21.15.26:0/1945373661 >> v2:172.21.15.93:3300/0 conn(0x7f40140b3d90 msgr2=0x7f401415faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 --2- 172.21.15.26:0/1945373661 >> v2:172.21.15.93:3300/0 conn(0x7f40140b3d90 0x7f401415faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 -- 172.21.15.26:0/1945373661 >> v1:172.21.15.93:6789/0 conn(0x7f40140726a0 legacy=0x7f4014072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 -- 172.21.15.26:0/1945373661 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401415eab0 con 0x7f4014073190 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.970+0000 7f4018dbb700 1 --2- 172.21.15.26:0/1945373661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 0x7f401415ffe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f400c009dc0 tx=0x7f400c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91b863a17a84c6c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f4012ffd700 1 -- 172.21.15.26:0/1945373661 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00e070 con 0x7f4014073190 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f4012ffd700 1 -- 172.21.15.26:0/1945373661 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c00bae0 con 0x7f4014073190 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f4012ffd700 1 -- 172.21.15.26:0/1945373661 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c00be90 con 0x7f4014073190 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 -- 172.21.15.26:0/1945373661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 msgr2=0x7f401415ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 --2- 172.21.15.26:0/1945373661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 0x7f401415ffe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f400c009dc0 tx=0x7f400c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 -- 172.21.15.26:0/1945373661 shutdown_connections 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 --2- 172.21.15.26:0/1945373661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4014073190 0x7f401415ffe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 --2- 172.21.15.26:0/1945373661 >> v2:172.21.15.93:3300/0 conn(0x7f40140b3d90 0x7f401415faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 -- 172.21.15.26:0/1945373661 >> 172.21.15.26:0/1945373661 conn(0x7f40140717f0 msgr2=0x7f40140b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 -- 172.21.15.26:0/1945373661 shutdown_connections 2024-01-16T14:55:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.971+0000 7f401a81e700 1 -- 172.21.15.26:0/1945373661 wait complete. 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.972+0000 7f401a81e700 1 Processor -- start 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.973+0000 7f401a81e700 1 -- start start 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.973+0000 7f401a81e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 0x7f401409dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.973+0000 7f401a81e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.973+0000 7f401a81e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40140b3d90 0x7f401415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.973+0000 7f401a81e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140ad630 con 0x7f4014073190 2024-01-16T14:55:43.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f401a81e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40140af8c0 con 0x7f40140726a0 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f401a81e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401406eda0 con 0x7f40140b3d90 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36786/0 (socket says 172.21.15.26:36786) 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 -- 172.21.15.26:0/234480402 learned_addr learned my addr 172.21.15.26:0/234480402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f4018dbb700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40140b3d90 0x7f401415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f4013fff700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 0x7f401409dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 -- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40140b3d90 msgr2=0x7f401415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40140b3d90 0x7f401415bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 -- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 msgr2=0x7f401409dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 0x7f401409dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 -- 172.21.15.26:0/234480402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400c009a70 con 0x7f4014073190 2024-01-16T14:55:43.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f4013fff700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 0x7f401409dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f40137fe700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9e0 tx=0x7f400800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008009710 con 0x7f4014073190 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400800c850 con 0x7f4014073190 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.974+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401409b860 con 0x7f4014073190 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.975+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800cbc0 con 0x7f4014073190 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.975+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40140af540 con 0x7f4014073190 2024-01-16T14:55:43.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.976+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401406efc0 con 0x7f4014073190 2024-01-16T14:55:43.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.976+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f400800cd20 con 0x7f4014073190 2024-01-16T14:55:43.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.977+0000 7f4010ff9700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 0x7f400006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:43.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.977+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4008089bd0 con 0x7f4014073190 2024-01-16T14:55:43.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.977+0000 7f4013fff700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 0x7f400006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:43.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.978+0000 7f4013fff700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 0x7f400006d810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4004007fd0 tx=0x7f4004009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:43.982+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f401406efc0 con 0x7f4014073190 2024-01-16T14:55:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.196+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40140ad770 con 0x7f4014073190 2024-01-16T14:55:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.197+0000 7f4010ff9700 1 -- 172.21.15.26:0/234480402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16509 (secure 0 0 0) 0x7f40140ad770 con 0x7f4014073190 2024-01-16T14:55:44.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:44.180 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":32,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:55:44.170624+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:55:40.975214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T14:55:41.589672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.199+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 msgr2=0x7f400006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.199+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 0x7f400006d810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4004007fd0 tx=0x7f4004009040 comp rx=0 tx=0).stop 2024-01-16T14:55:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.199+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 msgr2=0x7f4014147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.199+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f400800e9e0 tx=0x7f400800ecf0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 shutdown_connections 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f40140b3d90 0x7f401415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f40140726a0 0x7f401409dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f400006b350 0x7f400006d810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 --2- 172.21.15.26:0/234480402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4014073190 0x7f4014147a00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 >> 172.21.15.26:0/234480402 conn(0x7f40140717f0 msgr2=0x7f401406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 shutdown_connections 2024-01-16T14:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.200+0000 7f401a81e700 1 -- 172.21.15.26:0/234480402 wait complete. 2024-01-16T14:55:44.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-16T14:55:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.353+0000 7fd399177700 1 Processor -- start 2024-01-16T14:55:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.353+0000 7fd399177700 1 -- start start 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd399177700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd3940726a0 0x7fd394072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd399177700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd394072fc0 0x7fd39406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd399177700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39415f3e0 con 0x7fd3940b3d90 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd399177700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940affd0 con 0x7fd394072fc0 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd399177700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39415e7d0 con 0x7fd3940726a0 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd394072fc0 0x7fd39406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd394072fc0 0x7fd39406c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36796/0 (socket says 172.21.15.26:36796) 2024-01-16T14:55:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 -- 172.21.15.26:0/3593576428 learned_addr learned my addr 172.21.15.26:0/3593576428 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd392d9d700 1 --2- 172.21.15.26:0/3593576428 >> v2:172.21.15.93:3300/0 conn(0x7fd3940726a0 0x7fd394072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 -- 172.21.15.26:0/3593576428 >> v2:172.21.15.93:3300/0 conn(0x7fd3940726a0 msgr2=0x7fd394072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 --2- 172.21.15.26:0/3593576428 >> v2:172.21.15.93:3300/0 conn(0x7fd3940726a0 0x7fd394072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 -- 172.21.15.26:0/3593576428 >> v1:172.21.15.93:6789/0 conn(0x7fd3940b3d90 legacy=0x7fd39406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.354+0000 7fd39359e700 1 -- 172.21.15.26:0/3593576428 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39415eab0 con 0x7fd394072fc0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd39359e700 1 --2- 172.21.15.26:0/3593576428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39406c1d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009e50 tx=0x7fd38c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c347912e1210d42 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd391d9b700 1 -- 172.21.15.26:0/3593576428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00e070 con 0x7fd394072fc0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd391d9b700 1 -- 172.21.15.26:0/3593576428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38c00b8d0 con 0x7fd394072fc0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd391d9b700 1 -- 172.21.15.26:0/3593576428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c00bc80 con 0x7fd394072fc0 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 -- 172.21.15.26:0/3593576428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 msgr2=0x7fd39406c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 --2- 172.21.15.26:0/3593576428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39406c1d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009e50 tx=0x7fd38c00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 -- 172.21.15.26:0/3593576428 shutdown_connections 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 --2- 172.21.15.26:0/3593576428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39406c1d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 --2- 172.21.15.26:0/3593576428 >> v2:172.21.15.93:3300/0 conn(0x7fd3940726a0 0x7fd394072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 -- 172.21.15.26:0/3593576428 >> 172.21.15.26:0/3593576428 conn(0x7fd3940717f0 msgr2=0x7fd3940b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 -- 172.21.15.26:0/3593576428 shutdown_connections 2024-01-16T14:55:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.355+0000 7fd399177700 1 -- 172.21.15.26:0/3593576428 wait complete. 2024-01-16T14:55:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.357+0000 7fd399177700 1 Processor -- start 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.357+0000 7fd399177700 1 -- start start 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3940726a0 0x7fd394152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3940b3d90 0x7fd39409b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394157eb0 con 0x7fd394072fc0 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39415ae70 con 0x7fd3940726a0 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd399177700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3941575f0 con 0x7fd3940b3d90 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36806/0 (socket says 172.21.15.26:36806) 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 -- 172.21.15.26:0/1479606220 learned_addr learned my addr 172.21.15.26:0/1479606220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39359e700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3940b3d90 0x7fd39409b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd392d9d700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3940726a0 0x7fd394152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 -- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3940b3d90 msgr2=0x7fd39409b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3940b3d90 0x7fd39409b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 -- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3940726a0 msgr2=0x7fd394152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3940726a0 0x7fd394152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.358+0000 7fd39259c700 1 -- 172.21.15.26:0/1479606220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c009b30 con 0x7fd394072fc0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd39259c700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c4c0 tx=0x7fd38800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880052d0 con 0x7fd394072fc0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800ce90 con 0x7fd394072fc0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd394157860 con 0x7fd394072fc0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388006860 con 0x7fd394072fc0 2024-01-16T14:55:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.359+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39415b0d0 con 0x7fd394072fc0 2024-01-16T14:55:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.360+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394158060 con 0x7fd394072fc0 2024-01-16T14:55:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.360+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd388003680 con 0x7fd394072fc0 2024-01-16T14:55:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.361+0000 7fd37b7fe700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 0x7fd38006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.361+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd388088d00 con 0x7fd394072fc0 2024-01-16T14:55:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.361+0000 7fd392d9d700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 0x7fd38006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.361+0000 7fd392d9d700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 0x7fd38006d810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd384009b40 tx=0x7fd384009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.366+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd394158060 con 0x7fd394072fc0 2024-01-16T14:55:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:44.629+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd394158060 con 0x7fd394072fc0 2024-01-16T14:55:45.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.430+0000 7fd37b7fe700 1 -- 172.21.15.26:0/1479606220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd394158060 con 0x7fd394072fc0 2024-01-16T14:55:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 msgr2=0x7fd38006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 0x7fd38006d810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd384009b40 tx=0x7fd384009210 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 msgr2=0x7fd39415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c4c0 tx=0x7fd38800c7d0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 shutdown_connections 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3940b3d90 0x7fd39409b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3940726a0 0x7fd394152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd38006b350 0x7fd38006d810 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 --2- 172.21.15.26:0/1479606220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd394072fc0 0x7fd39415a930 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 >> 172.21.15.26:0/1479606220 conn(0x7fd3940717f0 msgr2=0x7fd3940ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 shutdown_connections 2024-01-16T14:55:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.434+0000 7fd399177700 1 -- 172.21.15.26:0/1479606220 wait complete. 2024-01-16T14:55:45.432 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-16T14:55:45.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-16T14:55:45.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.593+0000 7f650201b700 1 Processor -- start 2024-01-16T14:55:45.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.593+0000 7f650201b700 1 -- start start 2024-01-16T14:55:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f650201b700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f64fc0b3d90 0x7f64fc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f650201b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc15f3e0 con 0x7f64fc0726a0 2024-01-16T14:55:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f650201b700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc15e7d0 con 0x7f64fc073190 2024-01-16T14:55:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f650201b700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc0affd0 con 0x7f64fc0b3d90 2024-01-16T14:55:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f64fb7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f64fc0726a0 0x7f64fc072a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43928/0 (socket says 172.21.15.26:43928) 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f64fb7fe700 1 -- 172.21.15.26:0/920050219 learned_addr learned my addr 172.21.15.26:0/920050219 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f64faffd700 1 --2- 172.21.15.26:0/920050219 >> v2:172.21.15.93:3300/0 conn(0x7f64fc0b3d90 0x7f64fc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820217300 0 0) 0x7f64fc15f3e0 con 0x7f64fc0726a0 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.594+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e80035b0 con 0x7f64fc0726a0 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426795296 0 0) 0x7f64fc15e7d0 con 0x7f64fc073190 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64fc15f3e0 con 0x7f64fc073190 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64faffd700 1 -- 172.21.15.26:0/920050219 >> v1:172.21.15.93:6789/0 conn(0x7f64fc073190 legacy=0x7f64fc15ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64faffd700 1 -- 172.21.15.26:0/920050219 >> v1:172.21.15.26:6789/0 conn(0x7f64fc0726a0 legacy=0x7f64fc072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64faffd700 1 -- 172.21.15.26:0/920050219 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc15eab0 con 0x7f64fc0b3d90 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64faffd700 1 --2- 172.21.15.26:0/920050219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0b3d90 0x7f64fc15faa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009870 tx=0x7f64e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47fc037fc4a491ca server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011830 con 0x7f64fc0b3d90 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4011990 con 0x7f64fc0b3d90 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.595+0000 7f64fa7fc700 1 -- 172.21.15.26:0/920050219 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011d00 con 0x7f64fc0b3d90 2024-01-16T14:55:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 -- 172.21.15.26:0/920050219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0b3d90 msgr2=0x7f64fc15faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 --2- 172.21.15.26:0/920050219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0b3d90 0x7f64fc15faa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009870 tx=0x7f64e4009b80 comp rx=0 tx=0).stop 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 -- 172.21.15.26:0/920050219 shutdown_connections 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 --2- 172.21.15.26:0/920050219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0b3d90 0x7f64fc15faa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 -- 172.21.15.26:0/920050219 >> 172.21.15.26:0/920050219 conn(0x7f64fc0717f0 msgr2=0x7f64fc0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 -- 172.21.15.26:0/920050219 shutdown_connections 2024-01-16T14:55:45.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.596+0000 7f650201b700 1 -- 172.21.15.26:0/920050219 wait complete. 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.597+0000 7f650201b700 1 Processor -- start 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.597+0000 7f650201b700 1 -- start start 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0726a0 0x7f64fc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64fc073190 0x7f64fc1f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1f4ae0 con 0x7f64fc0b3d90 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1f47e0 con 0x7f64fc0726a0 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f650201b700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1530d0 con 0x7f64fc073190 2024-01-16T14:55:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41376/0 (socket says 172.21.15.26:41376) 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f64fbfff700 1 -- 172.21.15.26:0/2706877070 learned_addr learned my addr 172.21.15.26:0/2706877070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f64fb7fe700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0726a0 0x7f64fc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.598+0000 7f64faffd700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64fc073190 0x7f64fc1f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 -- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64fc073190 msgr2=0x7f64fc1f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64fc073190 0x7f64fc1f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 -- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0726a0 msgr2=0x7f64fc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0726a0 0x7f64fc15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 -- 172.21.15.26:0/2706877070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40094a0 con 0x7f64fc0b3d90 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64fbfff700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64f400cc20 tx=0x7f64f400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4003c00 con 0x7f64fc0b3d90 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4003d60 con 0x7f64fc0b3d90 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc15be60 con 0x7f64fc0b3d90 2024-01-16T14:55:45.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400aae0 con 0x7f64fc0b3d90 2024-01-16T14:55:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.599+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc15c420 con 0x7f64fc0b3d90 2024-01-16T14:55:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.600+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc075480 con 0x7f64fc0b3d90 2024-01-16T14:55:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.601+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f64f400ac40 con 0x7f64fc0b3d90 2024-01-16T14:55:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.601+0000 7f64f3fff700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 0x7f64dc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.601+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f4089b30 con 0x7f64fc0b3d90 2024-01-16T14:55:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.601+0000 7f64fb7fe700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 0x7f64dc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.602+0000 7f64fb7fe700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 0x7f64dc06d810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64ec007fd0 tx=0x7f64ec007dc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:45.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.606+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64f4054a30 con 0x7f64fc0b3d90 2024-01-16T14:55:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:45.861+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f64fc072c00 con 0x7f64fc0b3d90 2024-01-16T14:55:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.433+0000 7f64f3fff700 1 -- 172.21.15.26:0/2706877070 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f64f4058050 con 0x7f64fc0b3d90 2024-01-16T14:55:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.435+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 msgr2=0x7f64dc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.435+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 0x7f64dc06d810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64ec007fd0 tx=0x7f64ec007dc0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.435+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 msgr2=0x7f64fc1f3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.435+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64f400cc20 tx=0x7f64f400cf30 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 shutdown_connections 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f64dc06b350 0x7f64dc06d810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64fc0b3d90 0x7f64fc1f3ac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f64fc0726a0 0x7f64fc15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 --2- 172.21.15.26:0/2706877070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64fc073190 0x7f64fc1f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 >> 172.21.15.26:0/2706877070 conn(0x7f64fc0717f0 msgr2=0x7f64fc0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 shutdown_connections 2024-01-16T14:55:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.436+0000 7f650201b700 1 -- 172.21.15.26:0/2706877070 wait complete. 2024-01-16T14:55:46.434 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-16T14:55:46.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.592+0000 7f0d53f8e700 1 Processor -- start 2024-01-16T14:55:46.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 -- start start 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c15e000 0x7f0d4c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d4c06fe00 0x7f0d4c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c0cfd60 con 0x7f0d4c0d0ae0 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cf4b0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.593+0000 7f0d53f8e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c06f430 con 0x7f0d4c15e000 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d4c06fe00 0x7f0d4c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0d4c06fe00 0x7f0d4c0cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41392/0 (socket says 172.21.15.26:41392) 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 -- 172.21.15.26:0/393958130 learned_addr learned my addr 172.21.15.26:0/393958130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d51529700 1 --1- 172.21.15.26:0/393958130 >> v1:172.21.15.26:6789/0 conn(0x7f0d4c0d0ae0 0x7f0d4c06f6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43936/0 (socket says 172.21.15.26:43936) 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d51d2a700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c15e000 0x7f0d4c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 -- 172.21.15.26:0/393958130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c15e000 msgr2=0x7f0d4c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c15e000 0x7f0d4c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 -- 172.21.15.26:0/393958130 >> v1:172.21.15.26:6789/0 conn(0x7f0d4c0d0ae0 legacy=0x7f0d4c06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 -- 172.21.15.26:0/393958130 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0cf1d0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d50d28700 1 -- 172.21.15.26:0/393958130 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001378130 0 0) 0x7f0d4c0cfd60 con 0x7f0d4c0d0ae0 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d5252b700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c0cc220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009e50 tx=0x7f0d4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeb11b69a06e6fd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d50d28700 1 -- 172.21.15.26:0/393958130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400e070 con 0x7f0d4c06fe00 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d50d28700 1 -- 172.21.15.26:0/393958130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4400b910 con 0x7f0d4c06fe00 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.594+0000 7f0d50d28700 1 -- 172.21.15.26:0/393958130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4400bcc0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 -- 172.21.15.26:0/393958130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 msgr2=0x7f0d4c0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c0cc220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009e50 tx=0x7f0d4400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 -- 172.21.15.26:0/393958130 shutdown_connections 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c0cc220 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/393958130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c15e000 0x7f0d4c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 -- 172.21.15.26:0/393958130 >> 172.21.15.26:0/393958130 conn(0x7f0d4c0b3720 msgr2=0x7f0d4c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 -- 172.21.15.26:0/393958130 shutdown_connections 2024-01-16T14:55:46.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.595+0000 7f0d53f8e700 1 -- 172.21.15.26:0/393958130 wait complete. 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.597+0000 7f0d53f8e700 1 Processor -- start 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.597+0000 7f0d53f8e700 1 -- start start 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c0d0ae0 0x7f0d4c1f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d4c15e000 0x7f0d4c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c1f3780 con 0x7f0d4c06fe00 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c1591f0 con 0x7f0d4c15e000 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c160e20 con 0x7f0d4c0d0ae0 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41406/0 (socket says 172.21.15.26:41406) 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 -- 172.21.15.26:0/2612136516 learned_addr learned my addr 172.21.15.26:0/2612136516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51529700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c0d0ae0 0x7f0d4c1f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d5252b700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d4c15e000 0x7f0d4c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 -- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c0d0ae0 msgr2=0x7f0d4c1f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c0d0ae0 0x7f0d4c1f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 -- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d4c15e000 msgr2=0x7f0d4c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d4c15e000 0x7f0d4c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 -- 172.21.15.26:0/2612136516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009b30 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d51d2a700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9b0 tx=0x7f0d4800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48009710 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4800c850 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c161100 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.598+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800cbc0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.599+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c159770 con 0x7f0d4c06fe00 2024-01-16T14:55:46.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.599+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c159ae0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.600+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0d4800cd20 con 0x7f0d4c06fe00 2024-01-16T14:55:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.600+0000 7f0d427fc700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 0x7f0d3006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.601+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d48089c50 con 0x7f0d4c06fe00 2024-01-16T14:55:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.601+0000 7f0d51529700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 0x7f0d3006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.601+0000 7f0d51529700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 0x7f0d3006d810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c06e180 tx=0x7f0d3c007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.605+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d4808f020 con 0x7f0d4c06fe00 2024-01-16T14:55:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.862+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d4c0b43f0 con 0x7f0d4c06fe00 2024-01-16T14:55:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.863+0000 7f0d427fc700 1 -- 172.21.15.26:0/2612136516 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f0d48021090 con 0x7f0d4c06fe00 2024-01-16T14:55:46.845 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/923856271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923856271},{"type":"v1","addr":"172.21.15.26:6834","nonce":923856271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3873344155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3873344155},{"type":"v1","addr":"172.21.15.26:6835","nonce":3873344155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6836/396498543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":396498543},{"type":"v1","addr":"172.21.15.93:6836","nonce":396498543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:55:42.618646+0000","modified":"2024-01-16T14:55:45.863082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.93:6837/406936670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":406936670},{"type":"v1","addr":"172.21.15.93:6837","nonce":406936670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-16T14:55:46.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 msgr2=0x7f0d3006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 0x7f0d3006d810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c06e180 tx=0x7f0d3c007e70 comp rx=0 tx=0).stop 2024-01-16T14:55:46.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 msgr2=0x7f0d4c1f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:46.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800e9b0 tx=0x7f0d4800ecc0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 shutdown_connections 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d4c15e000 0x7f0d4c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f0d3006b350 0x7f0d3006d810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d4c06fe00 0x7f0d4c1f2d00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 --2- 172.21.15.26:0/2612136516 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d4c0d0ae0 0x7f0d4c1f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 >> 172.21.15.26:0/2612136516 conn(0x7f0d4c0b3720 msgr2=0x7f0d4c15d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 shutdown_connections 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:46.865+0000 7f0d53f8e700 1 -- 172.21.15.26:0/2612136516 wait complete. 2024-01-16T14:55:46.849 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-01-16T14:55:46.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:55:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.016+0000 7f6dfc130700 1 Processor -- start 2024-01-16T14:55:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.016+0000 7f6dfc130700 1 -- start start 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6dfc130700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6df40d2ba0 0x7f6df4168e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6dfc130700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df40d4e50 con 0x7f6df415e460 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6dfc130700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df40b3370 con 0x7f6df40d49d0 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6dfc130700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40cfc40 con 0x7f6df40d2ba0 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6df9ecc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6df415e460 0x7f6df40cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43938/0 (socket says 172.21.15.26:43938) 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6df9ecc700 1 -- 172.21.15.26:0/3850503202 learned_addr learned my addr 172.21.15.26:0/3850503202 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6dfa6cd700 1 --2- 172.21.15.26:0/3850503202 >> v2:172.21.15.93:3300/0 conn(0x7f6df40d2ba0 0x7f6df4168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.017+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024717516 0 0) 0x7f6df40d4e50 con 0x7f6df415e460 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de00035b0 con 0x7f6df415e460 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708714579 0 0) 0x7f6df40b3370 con 0x7f6df40d49d0 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df40d4e50 con 0x7f6df40d49d0 2024-01-16T14:55:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1386658946 0 0) 0x7f6de00035b0 con 0x7f6df415e460 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6dfa6cd700 1 -- 172.21.15.26:0/3850503202 >> v1:172.21.15.93:6789/0 conn(0x7f6df40d49d0 legacy=0x7f6df40d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6dfa6cd700 1 -- 172.21.15.26:0/3850503202 >> v1:172.21.15.26:6789/0 conn(0x7f6df415e460 legacy=0x7f6df40cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6dfa6cd700 1 -- 172.21.15.26:0/3850503202 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40b35c0 con 0x7f6df40d2ba0 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6dfa6cd700 1 --2- 172.21.15.26:0/3850503202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df40d2ba0 0x7f6df4168e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009d90 tx=0x7f6dec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85aea9fc857bad28 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00e070 con 0x7f6df40d2ba0 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec003a20 con 0x7f6df40d2ba0 2024-01-16T14:55:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.018+0000 7f6df8eca700 1 -- 172.21.15.26:0/3850503202 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0001670 con 0x7f6df40d2ba0 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 -- 172.21.15.26:0/3850503202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df40d2ba0 msgr2=0x7f6df4168e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3850503202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df40d2ba0 0x7f6df4168e40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009d90 tx=0x7f6dec00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 -- 172.21.15.26:0/3850503202 shutdown_connections 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3850503202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df40d2ba0 0x7f6df4168e40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 -- 172.21.15.26:0/3850503202 >> 172.21.15.26:0/3850503202 conn(0x7f6df40d3eb0 msgr2=0x7f6df40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 -- 172.21.15.26:0/3850503202 shutdown_connections 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.019+0000 7f6dfc130700 1 -- 172.21.15.26:0/3850503202 wait complete. 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.020+0000 7f6dfc130700 1 Processor -- start 2024-01-16T14:55:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.020+0000 7f6dfc130700 1 -- start start 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df40d2ba0 0x7f6df414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 0x7f6df40b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40a1c40 con 0x7f6df40d49d0 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df414e9c0 con 0x7f6df415e460 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfc130700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40a1680 con 0x7f6df40d2ba0 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41422/0 (socket says 172.21.15.26:41422) 2024-01-16T14:55:47.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 -- 172.21.15.26:0/3403210156 learned_addr learned my addr 172.21.15.26:0/3403210156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6dfa6cd700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 0x7f6df40b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df9ecc700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df40d2ba0 0x7f6df414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 -- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df40d2ba0 msgr2=0x7f6df414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df40d2ba0 0x7f6df414e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 -- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 msgr2=0x7f6df40b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.021+0000 7f6df96cb700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 0x7f6df40b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6df96cb700 1 -- 172.21.15.26:0/3403210156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009a70 con 0x7f6df40d49d0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6dfa6cd700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 0x7f6df40b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6df96cb700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c550 tx=0x7f6de800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80052d0 con 0x7f6df40d49d0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8005430 con 0x7f6df40d49d0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df40a1960 con 0x7f6df40d49d0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de80038c0 con 0x7f6df40d49d0 2024-01-16T14:55:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.022+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df40cc450 con 0x7f6df40d49d0 2024-01-16T14:55:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.023+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df40d3be0 con 0x7f6df40d49d0 2024-01-16T14:55:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.024+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6de8003a20 con 0x7f6df40d49d0 2024-01-16T14:55:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.024+0000 7f6de67fc700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 0x7f6dd806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.024+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6de8089070 con 0x7f6df40d49d0 2024-01-16T14:55:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.024+0000 7f6df9ecc700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 0x7f6dd806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.025+0000 7f6df9ecc700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 0x7f6dd806d810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6df0007fd0 tx=0x7f6df0007a60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.029+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6de8053f70 con 0x7f6df40d49d0 2024-01-16T14:55:47.131 INFO:tasks.ceph.mds.d.smithi093.stderr:2024-01-16T14:55:47.149+0000 7f56ae054700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-16T14:55:47.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.243+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6df4075480 con 0x7f6df40d49d0 2024-01-16T14:55:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.244+0000 7f6de67fc700 1 -- 172.21.15.26:0/3403210156 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16509 (secure 0 0 0) 0x7f6de8057590 con 0x7f6df40d49d0 2024-01-16T14:55:47.227 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:47.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":32,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:55:44.170624+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-16T14:55:40.975214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-16T14:55:41.589672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 msgr2=0x7f6dd806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 0x7f6dd806d810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6df0007fd0 tx=0x7f6df0007a60 comp rx=0 tx=0).stop 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 msgr2=0x7f6df40a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6de800c550 tx=0x7f6de800c910 comp rx=0 tx=0).stop 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 shutdown_connections 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6df415e460 0x7f6df40b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6df40d2ba0 0x7f6df414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6dd806b350 0x7f6dd806d810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 --2- 172.21.15.26:0/3403210156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df40d49d0 0x7f6df40a0d10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 >> 172.21.15.26:0/3403210156 conn(0x7f6df40d3eb0 msgr2=0x7f6df40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 shutdown_connections 2024-01-16T14:55:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.246+0000 7f6dfc130700 1 -- 172.21.15.26:0/3403210156 wait complete. 2024-01-16T14:55:47.243 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-16T14:55:47.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:55:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.398+0000 7fe8f109a700 1 Processor -- start 2024-01-16T14:55:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.398+0000 7fe8f109a700 1 -- start start 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8f109a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8f109a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe8ec06b520 0x7fe8ec06b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8f109a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec06b090 con 0x7fe8ec0b42f0 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8f109a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec06e900 con 0x7fe8ec06b520 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8f109a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec06ebe0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8ea59c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8eb59e700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe8ec06b520 0x7fe8ec06b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8eb59e700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe8ec06b520 0x7fe8ec06b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:50282/0 (socket says 172.21.15.26:50282) 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8ea59c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:40472/0 (socket says 172.21.15.26:40472) 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8eb59e700 1 -- 172.21.15.26:0/2826287539 learned_addr learned my addr 172.21.15.26:0/2826287539 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8ead9d700 1 --1- 172.21.15.26:0/2826287539 >> v1:172.21.15.93:6789/0 conn(0x7fe8ec0b42f0 0x7fe8ec0b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:39134/0 (socket says 172.21.15.26:39134) 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.399+0000 7fe8eb59e700 1 -- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 msgr2=0x7fe8ec06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8eb59e700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8eb59e700 1 -- 172.21.15.26:0/2826287539 >> v1:172.21.15.93:6789/0 conn(0x7fe8ec0b42f0 legacy=0x7fe8ec0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8eb59e700 1 -- 172.21.15.26:0/2826287539 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec06eec0 con 0x7fe8ec06b520 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8ea59c700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8e9d9b700 1 -- 172.21.15.26:0/2826287539 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312729600 0 0) 0x7fe8ec06b090 con 0x7fe8ec0b42f0 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8eb59e700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec06b9b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009e50 tx=0x7fe8e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcfee9b41349babb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8e9d9b700 1 -- 172.21.15.26:0/2826287539 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400e070 con 0x7fe8ec06b520 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8e9d9b700 1 -- 172.21.15.26:0/2826287539 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400b910 con 0x7fe8ec06b520 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8e9d9b700 1 -- 172.21.15.26:0/2826287539 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400bcc0 con 0x7fe8ec06b520 2024-01-16T14:55:47.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8f109a700 1 -- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 msgr2=0x7fe8ec06b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.400+0000 7fe8f109a700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec06b9b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009e50 tx=0x7fe8e400b330 comp rx=0 tx=0).stop 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 -- 172.21.15.26:0/2826287539 shutdown_connections 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec06b9b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 --2- 172.21.15.26:0/2826287539 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec15e4c0 0x7fe8ec06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 -- 172.21.15.26:0/2826287539 >> 172.21.15.26:0/2826287539 conn(0x7fe8ec0b24a0 msgr2=0x7fe8ec0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 -- 172.21.15.26:0/2826287539 shutdown_connections 2024-01-16T14:55:47.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.401+0000 7fe8f109a700 1 -- 172.21.15.26:0/2826287539 wait complete. 2024-01-16T14:55:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.402+0000 7fe8f109a700 1 Processor -- start 2024-01-16T14:55:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.402+0000 7fe8f109a700 1 -- start start 2024-01-16T14:55:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec0b42f0 0x7fe8ec1414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec148200 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec15b5c0 con 0x7fe8ec06b520 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8f109a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec15b7d0 con 0x7fe8ec0b42f0 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8eb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8ead9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8eb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41428/0 (socket says 172.21.15.26:41428) 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8eb59e700 1 -- 172.21.15.26:0/451495540 learned_addr learned my addr 172.21.15.26:0/451495540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.403+0000 7fe8ea59c700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec0b42f0 0x7fe8ec1414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 -- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec0b42f0 msgr2=0x7fe8ec1414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec0b42f0 0x7fe8ec1414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 -- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 msgr2=0x7fe8ec1446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec1446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 -- 172.21.15.26:0/451495540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e00094a0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8ead9d700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8eb59e700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400b680 tx=0x7fe8e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40048b0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e40099b0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e4009b30 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e401c680 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.404+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec1543c0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.405+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec0b66a0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.406+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe8e401c7e0 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.406+0000 7fe8d37fe700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 0x7fe8d806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.406+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8e408b420 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.407+0000 7fe8ead9d700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 0x7fe8d806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.407+0000 7fe8ead9d700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 0x7fe8d806d810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007fd0 tx=0x7fe8dc007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.411+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8e4056320 con 0x7fe8ec15e4c0 2024-01-16T14:55:47.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.619+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8ec06f1a0 con 0x7fe8d806b350 2024-01-16T14:55:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.628+0000 7fe8d37fe700 1 -- 172.21.15.26:0/451495540 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289703 (secure 0 0 0) 0x7fe8dc007140 con 0x7fe8d806b350 2024-01-16T14:55:47.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 msgr2=0x7fe8d806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 0x7fe8d806d810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007fd0 tx=0x7fe8dc007de0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 msgr2=0x7fe8ec147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400b680 tx=0x7fe8e4004ad0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 shutdown_connections 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fe8d806b350 0x7fe8d806d810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8ec15e4c0 0x7fe8ec147cc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8ec06b520 0x7fe8ec1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 --2- 172.21.15.26:0/451495540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe8ec0b42f0 0x7fe8ec1414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 >> 172.21.15.26:0/451495540 conn(0x7fe8ec0b24a0 msgr2=0x7fe8ec0b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:47.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 shutdown_connections 2024-01-16T14:55:47.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.632+0000 7fe8f109a700 1 -- 172.21.15.26:0/451495540 wait complete. 2024-01-16T14:55:47.618 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:55:47.629 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-01-16T14:55:47.364259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591130,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219992,"kb_used_data":3816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754728,"statfs":{"total":773094113280,"available":772868841472,"internally_reserved":0,"allocated":3907584,"data_stored":2213472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":762,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001159"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198760+0000","last_change":"2024-01-16T14:55:43.195812+0000","last_active":"2024-01-16T14:55:43.198760+0000","last_peered":"2024-01-16T14:55:43.198760+0000","last_clean":"2024-01-16T14:55:43.198760+0000","last_became_active":"2024-01-16T14:55:43.195556+0000","last_became_peered":"2024-01-16T14:55:43.195556+0000","last_unstale":"2024-01-16T14:55:43.198760+0000","last_undegraded":"2024-01-16T14:55:43.198760+0000","last_fullsized":"2024-01-16T14:55:43.198760+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:28.839366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198298+0000","last_change":"2024-01-16T14:55:43.194608+0000","last_active":"2024-01-16T14:55:43.198298+0000","last_peered":"2024-01-16T14:55:43.198298+0000","last_clean":"2024-01-16T14:55:43.198298+0000","last_became_active":"2024-01-16T14:55:43.194268+0000","last_became_peered":"2024-01-16T14:55:43.194268+0000","last_unstale":"2024-01-16T14:55:43.198298+0000","last_undegraded":"2024-01-16T14:55:43.198298+0000","last_fullsized":"2024-01-16T14:55:43.198298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:01:13.268844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201215+0000","last_change":"2024-01-16T14:55:43.198285+0000","last_active":"2024-01-16T14:55:43.201215+0000","last_peered":"2024-01-16T14:55:43.201215+0000","last_clean":"2024-01-16T14:55:43.201215+0000","last_became_active":"2024-01-16T14:55:43.197985+0000","last_became_peered":"2024-01-16T14:55:43.197985+0000","last_unstale":"2024-01-16T14:55:43.201215+0000","last_undegraded":"2024-01-16T14:55:43.201215+0000","last_fullsized":"2024-01-16T14:55:43.201215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.190981+0000","last_change":"2024-01-16T14:55:43.180438+0000","last_active":"2024-01-16T14:55:43.190981+0000","last_peered":"2024-01-16T14:55:43.190981+0000","last_clean":"2024-01-16T14:55:43.190981+0000","last_became_active":"2024-01-16T14:55:43.179418+0000","last_became_peered":"2024-01-16T14:55:43.179418+0000","last_unstale":"2024-01-16T14:55:43.190981+0000","last_undegraded":"2024-01-16T14:55:43.190981+0000","last_fullsized":"2024-01-16T14:55:43.190981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198787+0000","last_change":"2024-01-16T14:55:43.196956+0000","last_active":"2024-01-16T14:55:43.198787+0000","last_peered":"2024-01-16T14:55:43.198787+0000","last_clean":"2024-01-16T14:55:43.198787+0000","last_became_active":"2024-01-16T14:55:43.193831+0000","last_became_peered":"2024-01-16T14:55:43.193831+0000","last_unstale":"2024-01-16T14:55:43.198787+0000","last_undegraded":"2024-01-16T14:55:43.198787+0000","last_fullsized":"2024-01-16T14:55:43.198787+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:26:58.753714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.194203+0000","last_change":"2024-01-16T14:55:43.193204+0000","last_active":"2024-01-16T14:55:43.194203+0000","last_peered":"2024-01-16T14:55:43.194203+0000","last_clean":"2024-01-16T14:55:43.194203+0000","last_became_active":"2024-01-16T14:55:43.192861+0000","last_became_peered":"2024-01-16T14:55:43.192861+0000","last_unstale":"2024-01-16T14:55:43.194203+0000","last_undegraded":"2024-01-16T14:55:43.194203+0000","last_fullsized":"2024-01-16T14:55:43.194203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:06:09.820887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197084+0000","last_change":"2024-01-16T14:55:43.193837+0000","last_active":"2024-01-16T14:55:43.197084+0000","last_peered":"2024-01-16T14:55:43.197084+0000","last_clean":"2024-01-16T14:55:43.197084+0000","last_became_active":"2024-01-16T14:55:43.193502+0000","last_became_peered":"2024-01-16T14:55:43.193502+0000","last_unstale":"2024-01-16T14:55:43.197084+0000","last_undegraded":"2024-01-16T14:55:43.197084+0000","last_fullsized":"2024-01-16T14:55:43.197084+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196505+0000","last_change":"2024-01-16T14:55:43.191757+0000","last_active":"2024-01-16T14:55:43.196505+0000","last_peered":"2024-01-16T14:55:43.196505+0000","last_clean":"2024-01-16T14:55:43.196505+0000","last_became_active":"2024-01-16T14:55:43.191505+0000","last_became_peered":"2024-01-16T14:55:43.191505+0000","last_unstale":"2024-01-16T14:55:43.196505+0000","last_undegraded":"2024-01-16T14:55:43.196505+0000","last_fullsized":"2024-01-16T14:55:43.196505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188075+0000","last_change":"2024-01-16T14:55:43.182860+0000","last_active":"2024-01-16T14:55:43.188075+0000","last_peered":"2024-01-16T14:55:43.188075+0000","last_clean":"2024-01-16T14:55:43.188075+0000","last_became_active":"2024-01-16T14:55:43.182506+0000","last_became_peered":"2024-01-16T14:55:43.182506+0000","last_unstale":"2024-01-16T14:55:43.188075+0000","last_undegraded":"2024-01-16T14:55:43.188075+0000","last_fullsized":"2024-01-16T14:55:43.188075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:25:12.121075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195106+0000","last_change":"2024-01-16T14:55:43.185738+0000","last_active":"2024-01-16T14:55:43.195106+0000","last_peered":"2024-01-16T14:55:43.195106+0000","last_clean":"2024-01-16T14:55:43.195106+0000","last_became_active":"2024-01-16T14:55:43.185300+0000","last_became_peered":"2024-01-16T14:55:43.185300+0000","last_unstale":"2024-01-16T14:55:43.195106+0000","last_undegraded":"2024-01-16T14:55:43.195106+0000","last_fullsized":"2024-01-16T14:55:43.195106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198718+0000","last_change":"2024-01-16T14:55:43.196662+0000","last_active":"2024-01-16T14:55:43.198718+0000","last_peered":"2024-01-16T14:55:43.198718+0000","last_clean":"2024-01-16T14:55:43.198718+0000","last_became_active":"2024-01-16T14:55:43.196241+0000","last_became_peered":"2024-01-16T14:55:43.196241+0000","last_unstale":"2024-01-16T14:55:43.198718+0000","last_undegraded":"2024-01-16T14:55:43.198718+0000","last_fullsized":"2024-01-16T14:55:43.198718+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:59.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196555+0000","last_change":"2024-01-16T14:55:43.193570+0000","last_active":"2024-01-16T14:55:43.196555+0000","last_peered":"2024-01-16T14:55:43.196555+0000","last_clean":"2024-01-16T14:55:43.196555+0000","last_became_active":"2024-01-16T14:55:43.193285+0000","last_became_peered":"2024-01-16T14:55:43.193285+0000","last_unstale":"2024-01-16T14:55:43.196555+0000","last_undegraded":"2024-01-16T14:55:43.196555+0000","last_fullsized":"2024-01-16T14:55:43.196555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198807+0000","last_change":"2024-01-16T14:55:43.197210+0000","last_active":"2024-01-16T14:55:43.198807+0000","last_peered":"2024-01-16T14:55:43.198807+0000","last_clean":"2024-01-16T14:55:43.198807+0000","last_became_active":"2024-01-16T14:55:43.196869+0000","last_became_peered":"2024-01-16T14:55:43.196869+0000","last_unstale":"2024-01-16T14:55:43.198807+0000","last_undegraded":"2024-01-16T14:55:43.198807+0000","last_fullsized":"2024-01-16T14:55:43.198807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:44:44.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193380+0000","last_change":"2024-01-16T14:55:43.191433+0000","last_active":"2024-01-16T14:55:43.193380+0000","last_peered":"2024-01-16T14:55:43.193380+0000","last_clean":"2024-01-16T14:55:43.193380+0000","last_became_active":"2024-01-16T14:55:43.191050+0000","last_became_peered":"2024-01-16T14:55:43.191050+0000","last_unstale":"2024-01-16T14:55:43.193380+0000","last_undegraded":"2024-01-16T14:55:43.193380+0000","last_fullsized":"2024-01-16T14:55:43.193380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196017+0000","last_change":"2024-01-16T14:55:43.193575+0000","last_active":"2024-01-16T14:55:43.196017+0000","last_peered":"2024-01-16T14:55:43.196017+0000","last_clean":"2024-01-16T14:55:43.196017+0000","last_became_active":"2024-01-16T14:55:43.193301+0000","last_became_peered":"2024-01-16T14:55:43.193301+0000","last_unstale":"2024-01-16T14:55:43.196017+0000","last_undegraded":"2024-01-16T14:55:43.196017+0000","last_fullsized":"2024-01-16T14:55:43.196017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196466+0000","last_change":"2024-01-16T14:55:43.190813+0000","last_active":"2024-01-16T14:55:43.196466+0000","last_peered":"2024-01-16T14:55:43.196466+0000","last_clean":"2024-01-16T14:55:43.196466+0000","last_became_active":"2024-01-16T14:55:43.189860+0000","last_became_peered":"2024-01-16T14:55:43.189860+0000","last_unstale":"2024-01-16T14:55:43.196466+0000","last_undegraded":"2024-01-16T14:55:43.196466+0000","last_fullsized":"2024-01-16T14:55:43.196466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197205+0000","last_change":"2024-01-16T14:55:43.194194+0000","last_active":"2024-01-16T14:55:43.197205+0000","last_peered":"2024-01-16T14:55:43.197205+0000","last_clean":"2024-01-16T14:55:43.197205+0000","last_became_active":"2024-01-16T14:55:43.193808+0000","last_became_peered":"2024-01-16T14:55:43.193808+0000","last_unstale":"2024-01-16T14:55:43.197205+0000","last_undegraded":"2024-01-16T14:55:43.197205+0000","last_fullsized":"2024-01-16T14:55:43.197205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192317+0000","last_change":"2024-01-16T14:55:43.184531+0000","last_active":"2024-01-16T14:55:43.192317+0000","last_peered":"2024-01-16T14:55:43.192317+0000","last_clean":"2024-01-16T14:55:43.192317+0000","last_became_active":"2024-01-16T14:55:43.184138+0000","last_became_peered":"2024-01-16T14:55:43.184138+0000","last_unstale":"2024-01-16T14:55:43.192317+0000","last_undegraded":"2024-01-16T14:55:43.192317+0000","last_fullsized":"2024-01-16T14:55:43.192317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:23:55.486330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187451+0000","last_change":"2024-01-16T14:55:43.180484+0000","last_active":"2024-01-16T14:55:43.187451+0000","last_peered":"2024-01-16T14:55:43.187451+0000","last_clean":"2024-01-16T14:55:43.187451+0000","last_became_active":"2024-01-16T14:55:43.180167+0000","last_became_peered":"2024-01-16T14:55:43.180167+0000","last_unstale":"2024-01-16T14:55:43.187451+0000","last_undegraded":"2024-01-16T14:55:43.187451+0000","last_fullsized":"2024-01-16T14:55:43.187451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195267+0000","last_change":"2024-01-16T14:55:43.192957+0000","last_active":"2024-01-16T14:55:43.195267+0000","last_peered":"2024-01-16T14:55:43.195267+0000","last_clean":"2024-01-16T14:55:43.195267+0000","last_became_active":"2024-01-16T14:55:43.192624+0000","last_became_peered":"2024-01-16T14:55:43.192624+0000","last_unstale":"2024-01-16T14:55:43.195267+0000","last_undegraded":"2024-01-16T14:55:43.195267+0000","last_fullsized":"2024-01-16T14:55:43.195267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:48:48.768843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197397+0000","last_change":"2024-01-16T14:55:43.191210+0000","last_active":"2024-01-16T14:55:43.197397+0000","last_peered":"2024-01-16T14:55:43.197397+0000","last_clean":"2024-01-16T14:55:43.197397+0000","last_became_active":"2024-01-16T14:55:43.190898+0000","last_became_peered":"2024-01-16T14:55:43.190898+0000","last_unstale":"2024-01-16T14:55:43.197397+0000","last_undegraded":"2024-01-16T14:55:43.197397+0000","last_fullsized":"2024-01-16T14:55:43.197397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:42:26.976555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192794+0000","last_change":"2024-01-16T14:55:43.185949+0000","last_active":"2024-01-16T14:55:43.192794+0000","last_peered":"2024-01-16T14:55:43.192794+0000","last_clean":"2024-01-16T14:55:43.192794+0000","last_became_active":"2024-01-16T14:55:43.185657+0000","last_became_peered":"2024-01-16T14:55:43.185657+0000","last_unstale":"2024-01-16T14:55:43.192794+0000","last_undegraded":"2024-01-16T14:55:43.192794+0000","last_fullsized":"2024-01-16T14:55:43.192794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:33:21.628613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193594+0000","last_change":"2024-01-16T14:55:43.188874+0000","last_active":"2024-01-16T14:55:43.193594+0000","last_peered":"2024-01-16T14:55:43.193594+0000","last_clean":"2024-01-16T14:55:43.193594+0000","last_became_active":"2024-01-16T14:55:43.188547+0000","last_became_peered":"2024-01-16T14:55:43.188547+0000","last_unstale":"2024-01-16T14:55:43.193594+0000","last_undegraded":"2024-01-16T14:55:43.193594+0000","last_fullsized":"2024-01-16T14:55:43.193594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:28.839366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195907+0000","last_change":"2024-01-16T14:55:43.192325+0000","last_active":"2024-01-16T14:55:43.195907+0000","last_peered":"2024-01-16T14:55:43.195907+0000","last_clean":"2024-01-16T14:55:43.195907+0000","last_became_active":"2024-01-16T14:55:43.191903+0000","last_became_peered":"2024-01-16T14:55:43.191903+0000","last_unstale":"2024-01-16T14:55:43.195907+0000","last_undegraded":"2024-01-16T14:55:43.195907+0000","last_fullsized":"2024-01-16T14:55:43.195907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195294+0000","last_change":"2024-01-16T14:55:43.189139+0000","last_active":"2024-01-16T14:55:43.195294+0000","last_peered":"2024-01-16T14:55:43.195294+0000","last_clean":"2024-01-16T14:55:43.195294+0000","last_became_active":"2024-01-16T14:55:43.188826+0000","last_became_peered":"2024-01-16T14:55:43.188826+0000","last_unstale":"2024-01-16T14:55:43.195294+0000","last_undegraded":"2024-01-16T14:55:43.195294+0000","last_fullsized":"2024-01-16T14:55:43.195294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192723+0000","last_change":"2024-01-16T14:55:43.189390+0000","last_active":"2024-01-16T14:55:43.192723+0000","last_peered":"2024-01-16T14:55:43.192723+0000","last_clean":"2024-01-16T14:55:43.192723+0000","last_became_active":"2024-01-16T14:55:43.189085+0000","last_became_peered":"2024-01-16T14:55:43.189085+0000","last_unstale":"2024-01-16T14:55:43.192723+0000","last_undegraded":"2024-01-16T14:55:43.192723+0000","last_fullsized":"2024-01-16T14:55:43.192723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:54:20.701005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.190796+0000","last_change":"2024-01-16T14:55:43.179714+0000","last_active":"2024-01-16T14:55:43.190796+0000","last_peered":"2024-01-16T14:55:43.190796+0000","last_clean":"2024-01-16T14:55:43.190796+0000","last_became_active":"2024-01-16T14:55:43.179306+0000","last_became_peered":"2024-01-16T14:55:43.179306+0000","last_unstale":"2024-01-16T14:55:43.190796+0000","last_undegraded":"2024-01-16T14:55:43.190796+0000","last_fullsized":"2024-01-16T14:55:43.190796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:52.849751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197192+0000","last_change":"2024-01-16T14:55:43.194932+0000","last_active":"2024-01-16T14:55:43.197192+0000","last_peered":"2024-01-16T14:55:43.197192+0000","last_clean":"2024-01-16T14:55:43.197192+0000","last_became_active":"2024-01-16T14:55:43.194674+0000","last_became_peered":"2024-01-16T14:55:43.194674+0000","last_unstale":"2024-01-16T14:55:43.197192+0000","last_undegraded":"2024-01-16T14:55:43.197192+0000","last_fullsized":"2024-01-16T14:55:43.197192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:54:25.624931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192806+0000","last_change":"2024-01-16T14:55:43.185254+0000","last_active":"2024-01-16T14:55:43.192806+0000","last_peered":"2024-01-16T14:55:43.192806+0000","last_clean":"2024-01-16T14:55:43.192806+0000","last_became_active":"2024-01-16T14:55:43.184942+0000","last_became_peered":"2024-01-16T14:55:43.184942+0000","last_unstale":"2024-01-16T14:55:43.192806+0000","last_undegraded":"2024-01-16T14:55:43.192806+0000","last_fullsized":"2024-01-16T14:55:43.192806+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:59:47.307096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197174+0000","last_change":"2024-01-16T14:55:43.194349+0000","last_active":"2024-01-16T14:55:43.197174+0000","last_peered":"2024-01-16T14:55:43.197174+0000","last_clean":"2024-01-16T14:55:43.197174+0000","last_became_active":"2024-01-16T14:55:43.193956+0000","last_became_peered":"2024-01-16T14:55:43.193956+0000","last_unstale":"2024-01-16T14:55:43.197174+0000","last_undegraded":"2024-01-16T14:55:43.197174+0000","last_fullsized":"2024-01-16T14:55:43.197174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198597+0000","last_change":"2024-01-16T14:55:43.193105+0000","last_active":"2024-01-16T14:55:43.198597+0000","last_peered":"2024-01-16T14:55:43.198597+0000","last_clean":"2024-01-16T14:55:43.198597+0000","last_became_active":"2024-01-16T14:55:43.192824+0000","last_became_peered":"2024-01-16T14:55:43.192824+0000","last_unstale":"2024-01-16T14:55:43.198597+0000","last_undegraded":"2024-01-16T14:55:43.198597+0000","last_fullsized":"2024-01-16T14:55:43.198597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196486+0000","last_change":"2024-01-16T14:55:43.191071+0000","last_active":"2024-01-16T14:55:43.196486+0000","last_peered":"2024-01-16T14:55:43.196486+0000","last_clean":"2024-01-16T14:55:43.196486+0000","last_became_active":"2024-01-16T14:55:43.190517+0000","last_became_peered":"2024-01-16T14:55:43.190517+0000","last_unstale":"2024-01-16T14:55:43.196486+0000","last_undegraded":"2024-01-16T14:55:43.196486+0000","last_fullsized":"2024-01-16T14:55:43.196486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196017+0000","last_change":"2024-01-16T14:55:43.188560+0000","last_active":"2024-01-16T14:55:43.196017+0000","last_peered":"2024-01-16T14:55:43.196017+0000","last_clean":"2024-01-16T14:55:43.196017+0000","last_became_active":"2024-01-16T14:55:43.188186+0000","last_became_peered":"2024-01-16T14:55:43.188186+0000","last_unstale":"2024-01-16T14:55:43.196017+0000","last_undegraded":"2024-01-16T14:55:43.196017+0000","last_fullsized":"2024-01-16T14:55:43.196017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:52.849751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.183357+0000","last_change":"2024-01-16T14:55:42.191801+0000","last_active":"2024-01-16T14:55:43.183357+0000","last_peered":"2024-01-16T14:55:43.183357+0000","last_clean":"2024-01-16T14:55:43.183357+0000","last_became_active":"2024-01-16T14:55:42.191512+0000","last_became_peered":"2024-01-16T14:55:42.191512+0000","last_unstale":"2024-01-16T14:55:43.183357+0000","last_undegraded":"2024-01-16T14:55:43.183357+0000","last_fullsized":"2024-01-16T14:55:43.183357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:51.843541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196817+0000","last_change":"2024-01-16T14:55:43.189282+0000","last_active":"2024-01-16T14:55:43.196817+0000","last_peered":"2024-01-16T14:55:43.196817+0000","last_clean":"2024-01-16T14:55:43.196817+0000","last_became_active":"2024-01-16T14:55:43.189006+0000","last_became_peered":"2024-01-16T14:55:43.189006+0000","last_unstale":"2024-01-16T14:55:43.196817+0000","last_undegraded":"2024-01-16T14:55:43.196817+0000","last_fullsized":"2024-01-16T14:55:43.196817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:42.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186704+0000","last_change":"2024-01-16T14:55:42.190865+0000","last_active":"2024-01-16T14:55:43.186704+0000","last_peered":"2024-01-16T14:55:43.186704+0000","last_clean":"2024-01-16T14:55:43.186704+0000","last_became_active":"2024-01-16T14:55:42.190554+0000","last_became_peered":"2024-01-16T14:55:42.190554+0000","last_unstale":"2024-01-16T14:55:43.186704+0000","last_undegraded":"2024-01-16T14:55:43.186704+0000","last_fullsized":"2024-01-16T14:55:43.186704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192687+0000","last_change":"2024-01-16T14:55:43.184422+0000","last_active":"2024-01-16T14:55:43.192687+0000","last_peered":"2024-01-16T14:55:43.192687+0000","last_clean":"2024-01-16T14:55:43.192687+0000","last_became_active":"2024-01-16T14:55:43.184168+0000","last_became_peered":"2024-01-16T14:55:43.184168+0000","last_unstale":"2024-01-16T14:55:43.192687+0000","last_undegraded":"2024-01-16T14:55:43.192687+0000","last_fullsized":"2024-01-16T14:55:43.192687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:33.263332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175089+0000","last_change":"2024-01-16T14:55:42.191693+0000","last_active":"2024-01-16T14:55:43.175089+0000","last_peered":"2024-01-16T14:55:43.175089+0000","last_clean":"2024-01-16T14:55:43.175089+0000","last_became_active":"2024-01-16T14:55:42.191377+0000","last_became_peered":"2024-01-16T14:55:42.191377+0000","last_unstale":"2024-01-16T14:55:43.175089+0000","last_undegraded":"2024-01-16T14:55:43.175089+0000","last_fullsized":"2024-01-16T14:55:43.175089+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:20.715753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195431+0000","last_change":"2024-01-16T14:55:43.184609+0000","last_active":"2024-01-16T14:55:43.195431+0000","last_peered":"2024-01-16T14:55:43.195431+0000","last_clean":"2024-01-16T14:55:43.195431+0000","last_became_active":"2024-01-16T14:55:43.184325+0000","last_became_peered":"2024-01-16T14:55:43.184325+0000","last_unstale":"2024-01-16T14:55:43.195431+0000","last_undegraded":"2024-01-16T14:55:43.195431+0000","last_fullsized":"2024-01-16T14:55:43.195431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186761+0000","last_change":"2024-01-16T14:55:42.189119+0000","last_active":"2024-01-16T14:55:43.186761+0000","last_peered":"2024-01-16T14:55:43.186761+0000","last_clean":"2024-01-16T14:55:43.186761+0000","last_became_active":"2024-01-16T14:55:42.188886+0000","last_became_peered":"2024-01-16T14:55:42.188886+0000","last_unstale":"2024-01-16T14:55:43.186761+0000","last_undegraded":"2024-01-16T14:55:43.186761+0000","last_fullsized":"2024-01-16T14:55:43.186761+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:59:46.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192446+0000","last_change":"2024-01-16T14:55:43.188240+0000","last_active":"2024-01-16T14:55:43.192446+0000","last_peered":"2024-01-16T14:55:43.192446+0000","last_clean":"2024-01-16T14:55:43.192446+0000","last_became_active":"2024-01-16T14:55:43.187884+0000","last_became_peered":"2024-01-16T14:55:43.187884+0000","last_unstale":"2024-01-16T14:55:43.192446+0000","last_undegraded":"2024-01-16T14:55:43.192446+0000","last_fullsized":"2024-01-16T14:55:43.192446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176660+0000","last_change":"2024-01-16T14:55:42.182853+0000","last_active":"2024-01-16T14:55:43.176660+0000","last_peered":"2024-01-16T14:55:43.176660+0000","last_clean":"2024-01-16T14:55:43.176660+0000","last_became_active":"2024-01-16T14:55:42.182520+0000","last_became_peered":"2024-01-16T14:55:42.182520+0000","last_unstale":"2024-01-16T14:55:43.176660+0000","last_undegraded":"2024-01-16T14:55:43.176660+0000","last_fullsized":"2024-01-16T14:55:43.176660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:37.147610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201102+0000","last_change":"2024-01-16T14:55:43.197108+0000","last_active":"2024-01-16T14:55:43.201102+0000","last_peered":"2024-01-16T14:55:43.201102+0000","last_clean":"2024-01-16T14:55:43.201102+0000","last_became_active":"2024-01-16T14:55:43.196813+0000","last_became_peered":"2024-01-16T14:55:43.196813+0000","last_unstale":"2024-01-16T14:55:43.201102+0000","last_undegraded":"2024-01-16T14:55:43.201102+0000","last_fullsized":"2024-01-16T14:55:43.201102+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:59.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176394+0000","last_change":"2024-01-16T14:55:42.183013+0000","last_active":"2024-01-16T14:55:43.176394+0000","last_peered":"2024-01-16T14:55:43.176394+0000","last_clean":"2024-01-16T14:55:43.176394+0000","last_became_active":"2024-01-16T14:55:42.182703+0000","last_became_peered":"2024-01-16T14:55:42.182703+0000","last_unstale":"2024-01-16T14:55:43.176394+0000","last_undegraded":"2024-01-16T14:55:43.176394+0000","last_fullsized":"2024-01-16T14:55:43.176394+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:27.833157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.200867+0000","last_change":"2024-01-16T14:55:43.193837+0000","last_active":"2024-01-16T14:55:43.200867+0000","last_peered":"2024-01-16T14:55:43.200867+0000","last_clean":"2024-01-16T14:55:43.200867+0000","last_became_active":"2024-01-16T14:55:43.193563+0000","last_became_peered":"2024-01-16T14:55:43.193563+0000","last_unstale":"2024-01-16T14:55:43.200867+0000","last_undegraded":"2024-01-16T14:55:43.200867+0000","last_fullsized":"2024-01-16T14:55:43.200867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:57:21.003387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173463+0000","last_change":"2024-01-16T14:55:42.189207+0000","last_active":"2024-01-16T14:55:43.173463+0000","last_peered":"2024-01-16T14:55:43.173463+0000","last_clean":"2024-01-16T14:55:43.173463+0000","last_became_active":"2024-01-16T14:55:42.188884+0000","last_became_peered":"2024-01-16T14:55:42.188884+0000","last_unstale":"2024-01-16T14:55:43.173463+0000","last_undegraded":"2024-01-16T14:55:43.173463+0000","last_fullsized":"2024-01-16T14:55:43.173463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196212+0000","last_change":"2024-01-16T14:55:43.190160+0000","last_active":"2024-01-16T14:55:43.196212+0000","last_peered":"2024-01-16T14:55:43.196212+0000","last_clean":"2024-01-16T14:55:43.196212+0000","last_became_active":"2024-01-16T14:55:43.189580+0000","last_became_peered":"2024-01-16T14:55:43.189580+0000","last_unstale":"2024-01-16T14:55:43.196212+0000","last_undegraded":"2024-01-16T14:55:43.196212+0000","last_fullsized":"2024-01-16T14:55:43.196212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:21.027877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175851+0000","last_change":"2024-01-16T14:55:42.186227+0000","last_active":"2024-01-16T14:55:43.175851+0000","last_peered":"2024-01-16T14:55:43.175851+0000","last_clean":"2024-01-16T14:55:43.175851+0000","last_became_active":"2024-01-16T14:55:42.185936+0000","last_became_peered":"2024-01-16T14:55:42.185936+0000","last_unstale":"2024-01-16T14:55:43.175851+0000","last_undegraded":"2024-01-16T14:55:43.175851+0000","last_fullsized":"2024-01-16T14:55:43.175851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:09:15.337306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196849+0000","last_change":"2024-01-16T14:55:43.194724+0000","last_active":"2024-01-16T14:55:43.196849+0000","last_peered":"2024-01-16T14:55:43.196849+0000","last_clean":"2024-01-16T14:55:43.196849+0000","last_became_active":"2024-01-16T14:55:43.194312+0000","last_became_peered":"2024-01-16T14:55:43.194312+0000","last_unstale":"2024-01-16T14:55:43.196849+0000","last_undegraded":"2024-01-16T14:55:43.196849+0000","last_fullsized":"2024-01-16T14:55:43.196849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174257+0000","last_change":"2024-01-16T14:55:42.186300+0000","last_active":"2024-01-16T14:55:43.174257+0000","last_peered":"2024-01-16T14:55:43.174257+0000","last_clean":"2024-01-16T14:55:43.174257+0000","last_became_active":"2024-01-16T14:55:42.186016+0000","last_became_peered":"2024-01-16T14:55:42.186016+0000","last_unstale":"2024-01-16T14:55:43.174257+0000","last_undegraded":"2024-01-16T14:55:43.174257+0000","last_fullsized":"2024-01-16T14:55:43.174257+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:58.194435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187327+0000","last_change":"2024-01-16T14:55:43.180280+0000","last_active":"2024-01-16T14:55:43.187327+0000","last_peered":"2024-01-16T14:55:43.187327+0000","last_clean":"2024-01-16T14:55:43.187327+0000","last_became_active":"2024-01-16T14:55:43.179960+0000","last_became_peered":"2024-01-16T14:55:43.179960+0000","last_unstale":"2024-01-16T14:55:43.187327+0000","last_undegraded":"2024-01-16T14:55:43.187327+0000","last_fullsized":"2024-01-16T14:55:43.187327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186912+0000","last_change":"2024-01-16T14:55:42.189901+0000","last_active":"2024-01-16T14:55:43.186912+0000","last_peered":"2024-01-16T14:55:43.186912+0000","last_clean":"2024-01-16T14:55:43.186912+0000","last_became_active":"2024-01-16T14:55:42.189275+0000","last_became_peered":"2024-01-16T14:55:42.189275+0000","last_unstale":"2024-01-16T14:55:43.186912+0000","last_undegraded":"2024-01-16T14:55:43.186912+0000","last_fullsized":"2024-01-16T14:55:43.186912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192642+0000","last_change":"2024-01-16T14:55:43.189681+0000","last_active":"2024-01-16T14:55:43.192642+0000","last_peered":"2024-01-16T14:55:43.192642+0000","last_clean":"2024-01-16T14:55:43.192642+0000","last_became_active":"2024-01-16T14:55:43.189390+0000","last_became_peered":"2024-01-16T14:55:43.189390+0000","last_unstale":"2024-01-16T14:55:43.192642+0000","last_undegraded":"2024-01-16T14:55:43.192642+0000","last_fullsized":"2024-01-16T14:55:43.192642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188821+0000","last_change":"2024-01-16T14:55:42.189180+0000","last_active":"2024-01-16T14:55:43.188821+0000","last_peered":"2024-01-16T14:55:43.188821+0000","last_clean":"2024-01-16T14:55:43.188821+0000","last_became_active":"2024-01-16T14:55:42.188906+0000","last_became_peered":"2024-01-16T14:55:42.188906+0000","last_unstale":"2024-01-16T14:55:43.188821+0000","last_undegraded":"2024-01-16T14:55:43.188821+0000","last_fullsized":"2024-01-16T14:55:43.188821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:23:03.103944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201193+0000","last_change":"2024-01-16T14:55:43.198186+0000","last_active":"2024-01-16T14:55:43.201193+0000","last_peered":"2024-01-16T14:55:43.201193+0000","last_clean":"2024-01-16T14:55:43.201193+0000","last_became_active":"2024-01-16T14:55:43.197930+0000","last_became_peered":"2024-01-16T14:55:43.197930+0000","last_unstale":"2024-01-16T14:55:43.201193+0000","last_undegraded":"2024-01-16T14:55:43.201193+0000","last_fullsized":"2024-01-16T14:55:43.201193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'5","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174517+0000","last_change":"2024-01-16T14:55:42.188042+0000","last_active":"2024-01-16T14:55:43.174517+0000","last_peered":"2024-01-16T14:55:43.174517+0000","last_clean":"2024-01-16T14:55:43.174517+0000","last_became_active":"2024-01-16T14:55:42.187726+0000","last_became_peered":"2024-01-16T14:55:42.187726+0000","last_unstale":"2024-01-16T14:55:43.174517+0000","last_undegraded":"2024-01-16T14:55:43.174517+0000","last_fullsized":"2024-01-16T14:55:43.174517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:00.985588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197466+0000","last_change":"2024-01-16T14:55:43.194584+0000","last_active":"2024-01-16T14:55:43.197466+0000","last_peered":"2024-01-16T14:55:43.197466+0000","last_clean":"2024-01-16T14:55:43.197466+0000","last_became_active":"2024-01-16T14:55:43.194325+0000","last_became_peered":"2024-01-16T14:55:43.194325+0000","last_unstale":"2024-01-16T14:55:43.197466+0000","last_undegraded":"2024-01-16T14:55:43.197466+0000","last_fullsized":"2024-01-16T14:55:43.197466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176698+0000","last_change":"2024-01-16T14:55:42.186365+0000","last_active":"2024-01-16T14:55:43.176698+0000","last_peered":"2024-01-16T14:55:43.176698+0000","last_clean":"2024-01-16T14:55:43.176698+0000","last_became_active":"2024-01-16T14:55:42.186065+0000","last_became_peered":"2024-01-16T14:55:42.186065+0000","last_unstale":"2024-01-16T14:55:43.176698+0000","last_undegraded":"2024-01-16T14:55:43.176698+0000","last_fullsized":"2024-01-16T14:55:43.176698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192680+0000","last_change":"2024-01-16T14:55:43.189567+0000","last_active":"2024-01-16T14:55:43.192680+0000","last_peered":"2024-01-16T14:55:43.192680+0000","last_clean":"2024-01-16T14:55:43.192680+0000","last_became_active":"2024-01-16T14:55:43.189233+0000","last_became_peered":"2024-01-16T14:55:43.189233+0000","last_unstale":"2024-01-16T14:55:43.192680+0000","last_undegraded":"2024-01-16T14:55:43.192680+0000","last_fullsized":"2024-01-16T14:55:43.192680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:54:25.624931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173878+0000","last_change":"2024-01-16T14:55:42.196199+0000","last_active":"2024-01-16T14:55:43.173878+0000","last_peered":"2024-01-16T14:55:43.173878+0000","last_clean":"2024-01-16T14:55:43.173878+0000","last_became_active":"2024-01-16T14:55:42.195882+0000","last_became_peered":"2024-01-16T14:55:42.195882+0000","last_unstale":"2024-01-16T14:55:43.173878+0000","last_undegraded":"2024-01-16T14:55:43.173878+0000","last_fullsized":"2024-01-16T14:55:43.173878+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:58.194435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196443+0000","last_change":"2024-01-16T14:55:43.190160+0000","last_active":"2024-01-16T14:55:43.196443+0000","last_peered":"2024-01-16T14:55:43.196443+0000","last_clean":"2024-01-16T14:55:43.196443+0000","last_became_active":"2024-01-16T14:55:43.189576+0000","last_became_peered":"2024-01-16T14:55:43.189576+0000","last_unstale":"2024-01-16T14:55:43.196443+0000","last_undegraded":"2024-01-16T14:55:43.196443+0000","last_fullsized":"2024-01-16T14:55:43.196443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173557+0000","last_change":"2024-01-16T14:55:42.186853+0000","last_active":"2024-01-16T14:55:43.173557+0000","last_peered":"2024-01-16T14:55:43.173557+0000","last_clean":"2024-01-16T14:55:43.173557+0000","last_became_active":"2024-01-16T14:55:42.186564+0000","last_became_peered":"2024-01-16T14:55:42.186564+0000","last_unstale":"2024-01-16T14:55:43.173557+0000","last_undegraded":"2024-01-16T14:55:43.173557+0000","last_fullsized":"2024-01-16T14:55:43.173557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198621+0000","last_change":"2024-01-16T14:55:43.193616+0000","last_active":"2024-01-16T14:55:43.198621+0000","last_peered":"2024-01-16T14:55:43.198621+0000","last_clean":"2024-01-16T14:55:43.198621+0000","last_became_active":"2024-01-16T14:55:43.193357+0000","last_became_peered":"2024-01-16T14:55:43.193357+0000","last_unstale":"2024-01-16T14:55:43.198621+0000","last_undegraded":"2024-01-16T14:55:43.198621+0000","last_fullsized":"2024-01-16T14:55:43.198621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175588+0000","last_change":"2024-01-16T14:55:42.185458+0000","last_active":"2024-01-16T14:55:43.175588+0000","last_peered":"2024-01-16T14:55:43.175588+0000","last_clean":"2024-01-16T14:55:43.175588+0000","last_became_active":"2024-01-16T14:55:42.185109+0000","last_became_peered":"2024-01-16T14:55:42.185109+0000","last_unstale":"2024-01-16T14:55:43.175588+0000","last_undegraded":"2024-01-16T14:55:43.175588+0000","last_fullsized":"2024-01-16T14:55:43.175588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:23:03.103944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.200855+0000","last_change":"2024-01-16T14:55:43.192503+0000","last_active":"2024-01-16T14:55:43.200855+0000","last_peered":"2024-01-16T14:55:43.200855+0000","last_clean":"2024-01-16T14:55:43.200855+0000","last_became_active":"2024-01-16T14:55:43.192243+0000","last_became_peered":"2024-01-16T14:55:43.192243+0000","last_unstale":"2024-01-16T14:55:43.200855+0000","last_undegraded":"2024-01-16T14:55:43.200855+0000","last_fullsized":"2024-01-16T14:55:43.200855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:51.884099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173642+0000","last_change":"2024-01-16T14:55:42.186447+0000","last_active":"2024-01-16T14:55:43.173642+0000","last_peered":"2024-01-16T14:55:43.173642+0000","last_clean":"2024-01-16T14:55:43.173642+0000","last_became_active":"2024-01-16T14:55:42.186161+0000","last_became_peered":"2024-01-16T14:55:42.186161+0000","last_unstale":"2024-01-16T14:55:43.173642+0000","last_undegraded":"2024-01-16T14:55:43.173642+0000","last_fullsized":"2024-01-16T14:55:43.173642+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201146+0000","last_change":"2024-01-16T14:55:43.197986+0000","last_active":"2024-01-16T14:55:43.201146+0000","last_peered":"2024-01-16T14:55:43.201146+0000","last_clean":"2024-01-16T14:55:43.201146+0000","last_became_active":"2024-01-16T14:55:43.197600+0000","last_became_peered":"2024-01-16T14:55:43.197600+0000","last_unstale":"2024-01-16T14:55:43.201146+0000","last_undegraded":"2024-01-16T14:55:43.201146+0000","last_fullsized":"2024-01-16T14:55:43.201146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.177076+0000","last_change":"2024-01-16T14:55:42.191548+0000","last_active":"2024-01-16T14:55:43.177076+0000","last_peered":"2024-01-16T14:55:43.177076+0000","last_clean":"2024-01-16T14:55:43.177076+0000","last_became_active":"2024-01-16T14:55:42.191160+0000","last_became_peered":"2024-01-16T14:55:42.191160+0000","last_unstale":"2024-01-16T14:55:43.177076+0000","last_undegraded":"2024-01-16T14:55:43.177076+0000","last_fullsized":"2024-01-16T14:55:43.177076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195248+0000","last_change":"2024-01-16T14:55:43.182487+0000","last_active":"2024-01-16T14:55:43.195248+0000","last_peered":"2024-01-16T14:55:43.195248+0000","last_clean":"2024-01-16T14:55:43.195248+0000","last_became_active":"2024-01-16T14:55:43.182152+0000","last_became_peered":"2024-01-16T14:55:43.182152+0000","last_unstale":"2024-01-16T14:55:43.195248+0000","last_undegraded":"2024-01-16T14:55:43.195248+0000","last_fullsized":"2024-01-16T14:55:43.195248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.177184+0000","last_change":"2024-01-16T14:55:42.189643+0000","last_active":"2024-01-16T14:55:43.177184+0000","last_peered":"2024-01-16T14:55:43.177184+0000","last_clean":"2024-01-16T14:55:43.177184+0000","last_became_active":"2024-01-16T14:55:42.189344+0000","last_became_peered":"2024-01-16T14:55:42.189344+0000","last_unstale":"2024-01-16T14:55:43.177184+0000","last_undegraded":"2024-01-16T14:55:43.177184+0000","last_fullsized":"2024-01-16T14:55:43.177184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193489+0000","last_change":"2024-01-16T14:55:43.190839+0000","last_active":"2024-01-16T14:55:43.193489+0000","last_peered":"2024-01-16T14:55:43.193489+0000","last_clean":"2024-01-16T14:55:43.193489+0000","last_became_active":"2024-01-16T14:55:43.190493+0000","last_became_peered":"2024-01-16T14:55:43.190493+0000","last_unstale":"2024-01-16T14:55:43.193489+0000","last_undegraded":"2024-01-16T14:55:43.193489+0000","last_fullsized":"2024-01-16T14:55:43.193489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:04:32.590914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176514+0000","last_change":"2024-01-16T14:55:42.183357+0000","last_active":"2024-01-16T14:55:43.176514+0000","last_peered":"2024-01-16T14:55:43.176514+0000","last_clean":"2024-01-16T14:55:43.176514+0000","last_became_active":"2024-01-16T14:55:42.183015+0000","last_became_peered":"2024-01-16T14:55:42.183015+0000","last_unstale":"2024-01-16T14:55:43.176514+0000","last_undegraded":"2024-01-16T14:55:43.176514+0000","last_fullsized":"2024-01-16T14:55:43.176514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:51.843541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195946+0000","last_change":"2024-01-16T14:55:43.193474+0000","last_active":"2024-01-16T14:55:43.195946+0000","last_peered":"2024-01-16T14:55:43.195946+0000","last_clean":"2024-01-16T14:55:43.195946+0000","last_became_active":"2024-01-16T14:55:43.193179+0000","last_became_peered":"2024-01-16T14:55:43.193179+0000","last_unstale":"2024-01-16T14:55:43.195946+0000","last_undegraded":"2024-01-16T14:55:43.195946+0000","last_fullsized":"2024-01-16T14:55:43.195946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:25:12.024242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182085+0000","last_change":"2024-01-16T14:55:42.193921+0000","last_active":"2024-01-16T14:55:43.182085+0000","last_peered":"2024-01-16T14:55:43.182085+0000","last_clean":"2024-01-16T14:55:43.182085+0000","last_became_active":"2024-01-16T14:55:42.193454+0000","last_became_peered":"2024-01-16T14:55:42.193454+0000","last_unstale":"2024-01-16T14:55:43.182085+0000","last_undegraded":"2024-01-16T14:55:43.182085+0000","last_fullsized":"2024-01-16T14:55:43.182085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:37.147610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196827+0000","last_change":"2024-01-16T14:55:43.191599+0000","last_active":"2024-01-16T14:55:43.196827+0000","last_peered":"2024-01-16T14:55:43.196827+0000","last_clean":"2024-01-16T14:55:43.196827+0000","last_became_active":"2024-01-16T14:55:43.191229+0000","last_became_peered":"2024-01-16T14:55:43.191229+0000","last_unstale":"2024-01-16T14:55:43.196827+0000","last_undegraded":"2024-01-16T14:55:43.196827+0000","last_fullsized":"2024-01-16T14:55:43.196827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:26:17.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182847+0000","last_change":"2024-01-16T14:55:42.196610+0000","last_active":"2024-01-16T14:55:43.182847+0000","last_peered":"2024-01-16T14:55:43.182847+0000","last_clean":"2024-01-16T14:55:43.182847+0000","last_became_active":"2024-01-16T14:55:42.196315+0000","last_became_peered":"2024-01-16T14:55:42.196315+0000","last_unstale":"2024-01-16T14:55:43.182847+0000","last_undegraded":"2024-01-16T14:55:43.182847+0000","last_fullsized":"2024-01-16T14:55:43.182847+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:20.021667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196387+0000","last_change":"2024-01-16T14:55:43.194688+0000","last_active":"2024-01-16T14:55:43.196387+0000","last_peered":"2024-01-16T14:55:43.196387+0000","last_clean":"2024-01-16T14:55:43.196387+0000","last_became_active":"2024-01-16T14:55:43.193736+0000","last_became_peered":"2024-01-16T14:55:43.193736+0000","last_unstale":"2024-01-16T14:55:43.196387+0000","last_undegraded":"2024-01-16T14:55:43.196387+0000","last_fullsized":"2024-01-16T14:55:43.196387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:32:59.945357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176196+0000","last_change":"2024-01-16T14:55:42.196613+0000","last_active":"2024-01-16T14:55:43.176196+0000","last_peered":"2024-01-16T14:55:43.176196+0000","last_clean":"2024-01-16T14:55:43.176196+0000","last_became_active":"2024-01-16T14:55:42.196342+0000","last_became_peered":"2024-01-16T14:55:42.196342+0000","last_unstale":"2024-01-16T14:55:43.176196+0000","last_undegraded":"2024-01-16T14:55:43.176196+0000","last_fullsized":"2024-01-16T14:55:43.176196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:20.021667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197103+0000","last_change":"2024-01-16T14:55:43.193907+0000","last_active":"2024-01-16T14:55:43.197103+0000","last_peered":"2024-01-16T14:55:43.197103+0000","last_clean":"2024-01-16T14:55:43.197103+0000","last_became_active":"2024-01-16T14:55:43.193541+0000","last_became_peered":"2024-01-16T14:55:43.193541+0000","last_unstale":"2024-01-16T14:55:43.197103+0000","last_undegraded":"2024-01-16T14:55:43.197103+0000","last_fullsized":"2024-01-16T14:55:43.197103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:21.721962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188113+0000","last_change":"2024-01-16T14:55:42.189118+0000","last_active":"2024-01-16T14:55:43.188113+0000","last_peered":"2024-01-16T14:55:43.188113+0000","last_clean":"2024-01-16T14:55:43.188113+0000","last_became_active":"2024-01-16T14:55:42.188821+0000","last_became_peered":"2024-01-16T14:55:42.188821+0000","last_unstale":"2024-01-16T14:55:43.188113+0000","last_undegraded":"2024-01-16T14:55:43.188113+0000","last_fullsized":"2024-01-16T14:55:43.188113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:06:08.814677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195034+0000","last_change":"2024-01-16T14:55:43.192878+0000","last_active":"2024-01-16T14:55:43.195034+0000","last_peered":"2024-01-16T14:55:43.195034+0000","last_clean":"2024-01-16T14:55:43.195034+0000","last_became_active":"2024-01-16T14:55:43.192545+0000","last_became_peered":"2024-01-16T14:55:43.192545+0000","last_unstale":"2024-01-16T14:55:43.195034+0000","last_undegraded":"2024-01-16T14:55:43.195034+0000","last_fullsized":"2024-01-16T14:55:43.195034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:48:30.261767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173396+0000","last_change":"2024-01-16T14:55:42.199427+0000","last_active":"2024-01-16T14:55:43.173396+0000","last_peered":"2024-01-16T14:55:43.173396+0000","last_clean":"2024-01-16T14:55:43.173396+0000","last_became_active":"2024-01-16T14:55:42.199117+0000","last_became_peered":"2024-01-16T14:55:42.199117+0000","last_unstale":"2024-01-16T14:55:43.173396+0000","last_undegraded":"2024-01-16T14:55:43.173396+0000","last_fullsized":"2024-01-16T14:55:43.173396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:27.833157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201199+0000","last_change":"2024-01-16T14:55:43.198245+0000","last_active":"2024-01-16T14:55:43.201199+0000","last_peered":"2024-01-16T14:55:43.201199+0000","last_clean":"2024-01-16T14:55:43.201199+0000","last_became_active":"2024-01-16T14:55:43.197959+0000","last_became_peered":"2024-01-16T14:55:43.197959+0000","last_unstale":"2024-01-16T14:55:43.201199+0000","last_undegraded":"2024-01-16T14:55:43.201199+0000","last_fullsized":"2024-01-16T14:55:43.201199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:21.721962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187066+0000","last_change":"2024-01-16T14:55:42.189680+0000","last_active":"2024-01-16T14:55:43.187066+0000","last_peered":"2024-01-16T14:55:43.187066+0000","last_clean":"2024-01-16T14:55:43.187066+0000","last_became_active":"2024-01-16T14:55:42.188880+0000","last_became_peered":"2024-01-16T14:55:42.188880+0000","last_unstale":"2024-01-16T14:55:43.187066+0000","last_undegraded":"2024-01-16T14:55:43.187066+0000","last_fullsized":"2024-01-16T14:55:43.187066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:29.967413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198747+0000","last_change":"2024-01-16T14:55:43.197141+0000","last_active":"2024-01-16T14:55:43.198747+0000","last_peered":"2024-01-16T14:55:43.198747+0000","last_clean":"2024-01-16T14:55:43.198747+0000","last_became_active":"2024-01-16T14:55:43.196841+0000","last_became_peered":"2024-01-16T14:55:43.196841+0000","last_unstale":"2024-01-16T14:55:43.198747+0000","last_undegraded":"2024-01-16T14:55:43.198747+0000","last_fullsized":"2024-01-16T14:55:43.198747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:42.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174957+0000","last_change":"2024-01-16T14:55:42.192877+0000","last_active":"2024-01-16T14:55:43.174957+0000","last_peered":"2024-01-16T14:55:43.174957+0000","last_clean":"2024-01-16T14:55:43.174957+0000","last_became_active":"2024-01-16T14:55:42.192524+0000","last_became_peered":"2024-01-16T14:55:42.192524+0000","last_unstale":"2024-01-16T14:55:43.174957+0000","last_undegraded":"2024-01-16T14:55:43.174957+0000","last_fullsized":"2024-01-16T14:55:43.174957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:20.715753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196875+0000","last_change":"2024-01-16T14:55:43.190880+0000","last_active":"2024-01-16T14:55:43.196875+0000","last_peered":"2024-01-16T14:55:43.196875+0000","last_clean":"2024-01-16T14:55:43.196875+0000","last_became_active":"2024-01-16T14:55:43.190600+0000","last_became_peered":"2024-01-16T14:55:43.190600+0000","last_unstale":"2024-01-16T14:55:43.196875+0000","last_undegraded":"2024-01-16T14:55:43.196875+0000","last_fullsized":"2024-01-16T14:55:43.196875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:09:16.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182181+0000","last_change":"2024-01-16T14:55:42.194306+0000","last_active":"2024-01-16T14:55:43.182181+0000","last_peered":"2024-01-16T14:55:43.182181+0000","last_clean":"2024-01-16T14:55:43.182181+0000","last_became_active":"2024-01-16T14:55:42.193919+0000","last_became_peered":"2024-01-16T14:55:42.193919+0000","last_unstale":"2024-01-16T14:55:43.182181+0000","last_undegraded":"2024-01-16T14:55:43.182181+0000","last_fullsized":"2024-01-16T14:55:43.182181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198239+0000","last_change":"2024-01-16T14:55:43.195458+0000","last_active":"2024-01-16T14:55:43.198239+0000","last_peered":"2024-01-16T14:55:43.198239+0000","last_clean":"2024-01-16T14:55:43.198239+0000","last_became_active":"2024-01-16T14:55:43.194941+0000","last_became_peered":"2024-01-16T14:55:43.194941+0000","last_unstale":"2024-01-16T14:55:43.198239+0000","last_undegraded":"2024-01-16T14:55:43.198239+0000","last_fullsized":"2024-01-16T14:55:43.198239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:21.027877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173672+0000","last_change":"2024-01-16T14:55:42.201893+0000","last_active":"2024-01-16T14:55:43.173672+0000","last_peered":"2024-01-16T14:55:43.173672+0000","last_clean":"2024-01-16T14:55:43.173672+0000","last_became_active":"2024-01-16T14:55:42.201563+0000","last_became_peered":"2024-01-16T14:55:42.201563+0000","last_unstale":"2024-01-16T14:55:43.173672+0000","last_undegraded":"2024-01-16T14:55:43.173672+0000","last_fullsized":"2024-01-16T14:55:43.173672+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:29.967413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187319+0000","last_change":"2024-01-16T14:55:43.180136+0000","last_active":"2024-01-16T14:55:43.187319+0000","last_peered":"2024-01-16T14:55:43.187319+0000","last_clean":"2024-01-16T14:55:43.187319+0000","last_became_active":"2024-01-16T14:55:43.179819+0000","last_became_peered":"2024-01-16T14:55:43.179819+0000","last_unstale":"2024-01-16T14:55:43.187319+0000","last_undegraded":"2024-01-16T14:55:43.187319+0000","last_fullsized":"2024-01-16T14:55:43.187319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176182+0000","last_change":"2024-01-16T14:55:42.186333+0000","last_active":"2024-01-16T14:55:43.176182+0000","last_peered":"2024-01-16T14:55:43.176182+0000","last_clean":"2024-01-16T14:55:43.176182+0000","last_became_active":"2024-01-16T14:55:42.186037+0000","last_became_peered":"2024-01-16T14:55:42.186037+0000","last_unstale":"2024-01-16T14:55:43.176182+0000","last_undegraded":"2024-01-16T14:55:43.176182+0000","last_fullsized":"2024-01-16T14:55:43.176182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:41.099065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192763+0000","last_change":"2024-01-16T14:55:43.185473+0000","last_active":"2024-01-16T14:55:43.192763+0000","last_peered":"2024-01-16T14:55:43.192763+0000","last_clean":"2024-01-16T14:55:43.192763+0000","last_became_active":"2024-01-16T14:55:43.185167+0000","last_became_peered":"2024-01-16T14:55:43.185167+0000","last_unstale":"2024-01-16T14:55:43.192763+0000","last_undegraded":"2024-01-16T14:55:43.192763+0000","last_fullsized":"2024-01-16T14:55:43.192763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:24:53.534604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175764+0000","last_change":"2024-01-16T14:55:42.188428+0000","last_active":"2024-01-16T14:55:43.175764+0000","last_peered":"2024-01-16T14:55:43.175764+0000","last_clean":"2024-01-16T14:55:43.175764+0000","last_became_active":"2024-01-16T14:55:42.188123+0000","last_became_peered":"2024-01-16T14:55:42.188123+0000","last_unstale":"2024-01-16T14:55:43.175764+0000","last_undegraded":"2024-01-16T14:55:43.175764+0000","last_fullsized":"2024-01-16T14:55:43.175764+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:15.420678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196467+0000","last_change":"2024-01-16T14:55:43.191677+0000","last_active":"2024-01-16T14:55:43.196467+0000","last_peered":"2024-01-16T14:55:43.196467+0000","last_clean":"2024-01-16T14:55:43.196467+0000","last_became_active":"2024-01-16T14:55:43.191385+0000","last_became_peered":"2024-01-16T14:55:43.191385+0000","last_unstale":"2024-01-16T14:55:43.196467+0000","last_undegraded":"2024-01-16T14:55:43.196467+0000","last_fullsized":"2024-01-16T14:55:43.196467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182150+0000","last_change":"2024-01-16T14:55:42.191276+0000","last_active":"2024-01-16T14:55:43.182150+0000","last_peered":"2024-01-16T14:55:43.182150+0000","last_clean":"2024-01-16T14:55:43.182150+0000","last_became_active":"2024-01-16T14:55:42.190987+0000","last_became_peered":"2024-01-16T14:55:42.190987+0000","last_unstale":"2024-01-16T14:55:43.182150+0000","last_undegraded":"2024-01-16T14:55:43.182150+0000","last_fullsized":"2024-01-16T14:55:43.182150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:25:11.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-16T14:55:42.171320+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:55:42.171320+0000","last_peered":"2024-01-16T14:55:42.171320+0000","last_clean":"2024-01-16T14:55:42.171320+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:55:42.171320+0000","last_undegraded":"2024-01-16T14:55:42.171320+0000","last_fullsized":"2024-01-16T14:55:42.171320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":762,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":716382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70899999999999996}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.755}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.625}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":126014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":132170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.44},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.55000000000000004}]},{"osd":7,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.496},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":925696,"data_stored":715763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.313},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.5}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.629},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":125395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":1.1459999999999999}]},{"osd":1,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":2.1349999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":2.2759999999999998}]},{"osd":3,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":1.008},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":1.2110000000000001}]},{"osd":4,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.5},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.96599999999999997}]},{"osd":7,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.93400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":132170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35999999999999999}]},{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.315},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.40200000000000002}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.63},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.54100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:55:47.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-16T14:55:47.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.827+0000 7f03e279a700 1 Processor -- start 2024-01-16T14:55:47.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.827+0000 7f03e279a700 1 -- start start 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.827+0000 7f03e279a700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03dc0cc780 0x7f03dc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.827+0000 7f03e279a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f03dc0c6060 0x7f03dc0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03e279a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0c5bd0 con 0x7f03dc0d0250 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03e279a700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0cdb30 con 0x7f03dc0cc780 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03e279a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0cd570 con 0x7f03dc0c6060 2024-01-16T14:55:47.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03dc0cc780 0x7f03dc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03dc0cc780 0x7f03dc0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41436/0 (socket says 172.21.15.26:41436) 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 -- 172.21.15.26:0/256835235 learned_addr learned my addr 172.21.15.26:0/256835235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03e0d37700 1 --2- 172.21.15.26:0/256835235 >> v2:172.21.15.93:3300/0 conn(0x7f03dc0c6060 0x7f03dc0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 -- 172.21.15.26:0/256835235 >> v2:172.21.15.93:3300/0 conn(0x7f03dc0c6060 msgr2=0x7f03dc0c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 --2- 172.21.15.26:0/256835235 >> v2:172.21.15.93:3300/0 conn(0x7f03dc0c6060 0x7f03dc0c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 -- 172.21.15.26:0/256835235 >> v1:172.21.15.93:6789/0 conn(0x7f03dc0d0250 legacy=0x7f03dc0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 -- 172.21.15.26:0/256835235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0cd850 con 0x7f03dc0cc780 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03db7fe700 1 --2- 172.21.15.26:0/256835235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc0ccc00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f03dc067980 tx=0x7f03d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8282cde8a92fd01f server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.828+0000 7f03daffd700 1 -- 172.21.15.26:0/256835235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011830 con 0x7f03dc0cc780 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03daffd700 1 -- 172.21.15.26:0/256835235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0011990 con 0x7f03dc0cc780 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03daffd700 1 -- 172.21.15.26:0/256835235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0011d20 con 0x7f03dc0cc780 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 -- 172.21.15.26:0/256835235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 msgr2=0x7f03dc0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 --2- 172.21.15.26:0/256835235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc0ccc00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f03dc067980 tx=0x7f03d0009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 -- 172.21.15.26:0/256835235 shutdown_connections 2024-01-16T14:55:47.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 --2- 172.21.15.26:0/256835235 >> v2:172.21.15.93:3300/0 conn(0x7f03dc0c6060 0x7f03dc0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 --2- 172.21.15.26:0/256835235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc0ccc00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 -- 172.21.15.26:0/256835235 >> 172.21.15.26:0/256835235 conn(0x7f03dc0b3800 msgr2=0x7f03dc0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:47.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 -- 172.21.15.26:0/256835235 shutdown_connections 2024-01-16T14:55:47.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.829+0000 7f03e279a700 1 -- 172.21.15.26:0/256835235 wait complete. 2024-01-16T14:55:47.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.830+0000 7f03e279a700 1 Processor -- start 2024-01-16T14:55:47.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.831+0000 7f03e279a700 1 -- start start 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.831+0000 7f03e279a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03dc0c6060 0x7f03dc15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.831+0000 7f03e279a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03dc0d0250 0x7f03dc09ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc15a860 con 0x7f03dc0cc780 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc1f8d40 con 0x7f03dc0c6060 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc1f80c0 con 0x7f03dc0d0250 2024-01-16T14:55:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41450/0 (socket says 172.21.15.26:41450) 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 -- 172.21.15.26:0/1774143924 learned_addr learned my addr 172.21.15.26:0/1774143924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03dbfff700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03dc0c6060 0x7f03dc15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e0d37700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03dc0d0250 0x7f03dc09ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 -- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03dc0d0250 msgr2=0x7f03dc09ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03dc0d0250 0x7f03dc09ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 -- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03dc0c6060 msgr2=0x7f03dc15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03dc0c6060 0x7f03dc15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 -- 172.21.15.26:0/1774143924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d0009550 con 0x7f03dc0cc780 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03db7fe700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f03d0000c00 tx=0x7f03d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0004380 con 0x7f03dc0cc780 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d001f070 con 0x7f03dc0cc780 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc1f83a0 con 0x7f03dc0cc780 2024-01-16T14:55:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00036a0 con 0x7f03dc0cc780 2024-01-16T14:55:47.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.832+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc1f8960 con 0x7f03dc0cc780 2024-01-16T14:55:47.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.833+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc1f8af0 con 0x7f03dc0cc780 2024-01-16T14:55:47.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.834+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f03d0021030 con 0x7f03dc0cc780 2024-01-16T14:55:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.834+0000 7f03d8ff9700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 0x7f03c806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.835+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03d008ae70 con 0x7f03dc0cc780 2024-01-16T14:55:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.835+0000 7f03dbfff700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 0x7f03c806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.835+0000 7f03dbfff700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 0x7f03c806d810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:47.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:47.840+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03d0055e90 con 0x7f03dc0cc780 2024-01-16T14:55:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.044+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03dc15c220 con 0x7f03c806b350 2024-01-16T14:55:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.053+0000 7f03d8ff9700 1 -- 172.21.15.26:0/1774143924 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289703 (secure 0 0 0) 0x7f03cc001cc0 con 0x7f03c806b350 2024-01-16T14:55:48.039 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 msgr2=0x7f03c806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 0x7f03c806d810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc009040 comp rx=0 tx=0).stop 2024-01-16T14:55:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 msgr2=0x7f03dc09c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f03d0000c00 tx=0x7f03d0004750 comp rx=0 tx=0).stop 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 shutdown_connections 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03dc0d0250 0x7f03dc09ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03dc0c6060 0x7f03dc15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c806b350 0x7f03c806d810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 --2- 172.21.15.26:0/1774143924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03dc0cc780 0x7f03dc09c530 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 >> 172.21.15.26:0/1774143924 conn(0x7f03dc0b3800 msgr2=0x7f03dc15e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.056+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 shutdown_connections 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.057+0000 7f03e279a700 1 -- 172.21.15.26:0/1774143924 wait complete. 2024-01-16T14:55:48.042 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-16T14:55:48.053 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-01-16T14:55:47.364259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591130,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":183,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219992,"kb_used_data":3816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754728,"statfs":{"total":773094113280,"available":772868841472,"internally_reserved":0,"allocated":3907584,"data_stored":2213472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":762,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001159"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198760+0000","last_change":"2024-01-16T14:55:43.195812+0000","last_active":"2024-01-16T14:55:43.198760+0000","last_peered":"2024-01-16T14:55:43.198760+0000","last_clean":"2024-01-16T14:55:43.198760+0000","last_became_active":"2024-01-16T14:55:43.195556+0000","last_became_peered":"2024-01-16T14:55:43.195556+0000","last_unstale":"2024-01-16T14:55:43.198760+0000","last_undegraded":"2024-01-16T14:55:43.198760+0000","last_fullsized":"2024-01-16T14:55:43.198760+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:28.839366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198298+0000","last_change":"2024-01-16T14:55:43.194608+0000","last_active":"2024-01-16T14:55:43.198298+0000","last_peered":"2024-01-16T14:55:43.198298+0000","last_clean":"2024-01-16T14:55:43.198298+0000","last_became_active":"2024-01-16T14:55:43.194268+0000","last_became_peered":"2024-01-16T14:55:43.194268+0000","last_unstale":"2024-01-16T14:55:43.198298+0000","last_undegraded":"2024-01-16T14:55:43.198298+0000","last_fullsized":"2024-01-16T14:55:43.198298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:01:13.268844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201215+0000","last_change":"2024-01-16T14:55:43.198285+0000","last_active":"2024-01-16T14:55:43.201215+0000","last_peered":"2024-01-16T14:55:43.201215+0000","last_clean":"2024-01-16T14:55:43.201215+0000","last_became_active":"2024-01-16T14:55:43.197985+0000","last_became_peered":"2024-01-16T14:55:43.197985+0000","last_unstale":"2024-01-16T14:55:43.201215+0000","last_undegraded":"2024-01-16T14:55:43.201215+0000","last_fullsized":"2024-01-16T14:55:43.201215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.190981+0000","last_change":"2024-01-16T14:55:43.180438+0000","last_active":"2024-01-16T14:55:43.190981+0000","last_peered":"2024-01-16T14:55:43.190981+0000","last_clean":"2024-01-16T14:55:43.190981+0000","last_became_active":"2024-01-16T14:55:43.179418+0000","last_became_peered":"2024-01-16T14:55:43.179418+0000","last_unstale":"2024-01-16T14:55:43.190981+0000","last_undegraded":"2024-01-16T14:55:43.190981+0000","last_fullsized":"2024-01-16T14:55:43.190981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198787+0000","last_change":"2024-01-16T14:55:43.196956+0000","last_active":"2024-01-16T14:55:43.198787+0000","last_peered":"2024-01-16T14:55:43.198787+0000","last_clean":"2024-01-16T14:55:43.198787+0000","last_became_active":"2024-01-16T14:55:43.193831+0000","last_became_peered":"2024-01-16T14:55:43.193831+0000","last_unstale":"2024-01-16T14:55:43.198787+0000","last_undegraded":"2024-01-16T14:55:43.198787+0000","last_fullsized":"2024-01-16T14:55:43.198787+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T01:26:58.753714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.194203+0000","last_change":"2024-01-16T14:55:43.193204+0000","last_active":"2024-01-16T14:55:43.194203+0000","last_peered":"2024-01-16T14:55:43.194203+0000","last_clean":"2024-01-16T14:55:43.194203+0000","last_became_active":"2024-01-16T14:55:43.192861+0000","last_became_peered":"2024-01-16T14:55:43.192861+0000","last_unstale":"2024-01-16T14:55:43.194203+0000","last_undegraded":"2024-01-16T14:55:43.194203+0000","last_fullsized":"2024-01-16T14:55:43.194203+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:06:09.820887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197084+0000","last_change":"2024-01-16T14:55:43.193837+0000","last_active":"2024-01-16T14:55:43.197084+0000","last_peered":"2024-01-16T14:55:43.197084+0000","last_clean":"2024-01-16T14:55:43.197084+0000","last_became_active":"2024-01-16T14:55:43.193502+0000","last_became_peered":"2024-01-16T14:55:43.193502+0000","last_unstale":"2024-01-16T14:55:43.197084+0000","last_undegraded":"2024-01-16T14:55:43.197084+0000","last_fullsized":"2024-01-16T14:55:43.197084+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196505+0000","last_change":"2024-01-16T14:55:43.191757+0000","last_active":"2024-01-16T14:55:43.196505+0000","last_peered":"2024-01-16T14:55:43.196505+0000","last_clean":"2024-01-16T14:55:43.196505+0000","last_became_active":"2024-01-16T14:55:43.191505+0000","last_became_peered":"2024-01-16T14:55:43.191505+0000","last_unstale":"2024-01-16T14:55:43.196505+0000","last_undegraded":"2024-01-16T14:55:43.196505+0000","last_fullsized":"2024-01-16T14:55:43.196505+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188075+0000","last_change":"2024-01-16T14:55:43.182860+0000","last_active":"2024-01-16T14:55:43.188075+0000","last_peered":"2024-01-16T14:55:43.188075+0000","last_clean":"2024-01-16T14:55:43.188075+0000","last_became_active":"2024-01-16T14:55:43.182506+0000","last_became_peered":"2024-01-16T14:55:43.182506+0000","last_unstale":"2024-01-16T14:55:43.188075+0000","last_undegraded":"2024-01-16T14:55:43.188075+0000","last_fullsized":"2024-01-16T14:55:43.188075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:25:12.121075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195106+0000","last_change":"2024-01-16T14:55:43.185738+0000","last_active":"2024-01-16T14:55:43.195106+0000","last_peered":"2024-01-16T14:55:43.195106+0000","last_clean":"2024-01-16T14:55:43.195106+0000","last_became_active":"2024-01-16T14:55:43.185300+0000","last_became_peered":"2024-01-16T14:55:43.185300+0000","last_unstale":"2024-01-16T14:55:43.195106+0000","last_undegraded":"2024-01-16T14:55:43.195106+0000","last_fullsized":"2024-01-16T14:55:43.195106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198718+0000","last_change":"2024-01-16T14:55:43.196662+0000","last_active":"2024-01-16T14:55:43.198718+0000","last_peered":"2024-01-16T14:55:43.198718+0000","last_clean":"2024-01-16T14:55:43.198718+0000","last_became_active":"2024-01-16T14:55:43.196241+0000","last_became_peered":"2024-01-16T14:55:43.196241+0000","last_unstale":"2024-01-16T14:55:43.198718+0000","last_undegraded":"2024-01-16T14:55:43.198718+0000","last_fullsized":"2024-01-16T14:55:43.198718+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:59.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196555+0000","last_change":"2024-01-16T14:55:43.193570+0000","last_active":"2024-01-16T14:55:43.196555+0000","last_peered":"2024-01-16T14:55:43.196555+0000","last_clean":"2024-01-16T14:55:43.196555+0000","last_became_active":"2024-01-16T14:55:43.193285+0000","last_became_peered":"2024-01-16T14:55:43.193285+0000","last_unstale":"2024-01-16T14:55:43.196555+0000","last_undegraded":"2024-01-16T14:55:43.196555+0000","last_fullsized":"2024-01-16T14:55:43.196555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198807+0000","last_change":"2024-01-16T14:55:43.197210+0000","last_active":"2024-01-16T14:55:43.198807+0000","last_peered":"2024-01-16T14:55:43.198807+0000","last_clean":"2024-01-16T14:55:43.198807+0000","last_became_active":"2024-01-16T14:55:43.196869+0000","last_became_peered":"2024-01-16T14:55:43.196869+0000","last_unstale":"2024-01-16T14:55:43.198807+0000","last_undegraded":"2024-01-16T14:55:43.198807+0000","last_fullsized":"2024-01-16T14:55:43.198807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:44:44.156072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193380+0000","last_change":"2024-01-16T14:55:43.191433+0000","last_active":"2024-01-16T14:55:43.193380+0000","last_peered":"2024-01-16T14:55:43.193380+0000","last_clean":"2024-01-16T14:55:43.193380+0000","last_became_active":"2024-01-16T14:55:43.191050+0000","last_became_peered":"2024-01-16T14:55:43.191050+0000","last_unstale":"2024-01-16T14:55:43.193380+0000","last_undegraded":"2024-01-16T14:55:43.193380+0000","last_fullsized":"2024-01-16T14:55:43.193380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196017+0000","last_change":"2024-01-16T14:55:43.193575+0000","last_active":"2024-01-16T14:55:43.196017+0000","last_peered":"2024-01-16T14:55:43.196017+0000","last_clean":"2024-01-16T14:55:43.196017+0000","last_became_active":"2024-01-16T14:55:43.193301+0000","last_became_peered":"2024-01-16T14:55:43.193301+0000","last_unstale":"2024-01-16T14:55:43.196017+0000","last_undegraded":"2024-01-16T14:55:43.196017+0000","last_fullsized":"2024-01-16T14:55:43.196017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196466+0000","last_change":"2024-01-16T14:55:43.190813+0000","last_active":"2024-01-16T14:55:43.196466+0000","last_peered":"2024-01-16T14:55:43.196466+0000","last_clean":"2024-01-16T14:55:43.196466+0000","last_became_active":"2024-01-16T14:55:43.189860+0000","last_became_peered":"2024-01-16T14:55:43.189860+0000","last_unstale":"2024-01-16T14:55:43.196466+0000","last_undegraded":"2024-01-16T14:55:43.196466+0000","last_fullsized":"2024-01-16T14:55:43.196466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197205+0000","last_change":"2024-01-16T14:55:43.194194+0000","last_active":"2024-01-16T14:55:43.197205+0000","last_peered":"2024-01-16T14:55:43.197205+0000","last_clean":"2024-01-16T14:55:43.197205+0000","last_became_active":"2024-01-16T14:55:43.193808+0000","last_became_peered":"2024-01-16T14:55:43.193808+0000","last_unstale":"2024-01-16T14:55:43.197205+0000","last_undegraded":"2024-01-16T14:55:43.197205+0000","last_fullsized":"2024-01-16T14:55:43.197205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192317+0000","last_change":"2024-01-16T14:55:43.184531+0000","last_active":"2024-01-16T14:55:43.192317+0000","last_peered":"2024-01-16T14:55:43.192317+0000","last_clean":"2024-01-16T14:55:43.192317+0000","last_became_active":"2024-01-16T14:55:43.184138+0000","last_became_peered":"2024-01-16T14:55:43.184138+0000","last_unstale":"2024-01-16T14:55:43.192317+0000","last_undegraded":"2024-01-16T14:55:43.192317+0000","last_fullsized":"2024-01-16T14:55:43.192317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:23:55.486330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187451+0000","last_change":"2024-01-16T14:55:43.180484+0000","last_active":"2024-01-16T14:55:43.187451+0000","last_peered":"2024-01-16T14:55:43.187451+0000","last_clean":"2024-01-16T14:55:43.187451+0000","last_became_active":"2024-01-16T14:55:43.180167+0000","last_became_peered":"2024-01-16T14:55:43.180167+0000","last_unstale":"2024-01-16T14:55:43.187451+0000","last_undegraded":"2024-01-16T14:55:43.187451+0000","last_fullsized":"2024-01-16T14:55:43.187451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195267+0000","last_change":"2024-01-16T14:55:43.192957+0000","last_active":"2024-01-16T14:55:43.195267+0000","last_peered":"2024-01-16T14:55:43.195267+0000","last_clean":"2024-01-16T14:55:43.195267+0000","last_became_active":"2024-01-16T14:55:43.192624+0000","last_became_peered":"2024-01-16T14:55:43.192624+0000","last_unstale":"2024-01-16T14:55:43.195267+0000","last_undegraded":"2024-01-16T14:55:43.195267+0000","last_fullsized":"2024-01-16T14:55:43.195267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:48:48.768843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197397+0000","last_change":"2024-01-16T14:55:43.191210+0000","last_active":"2024-01-16T14:55:43.197397+0000","last_peered":"2024-01-16T14:55:43.197397+0000","last_clean":"2024-01-16T14:55:43.197397+0000","last_became_active":"2024-01-16T14:55:43.190898+0000","last_became_peered":"2024-01-16T14:55:43.190898+0000","last_unstale":"2024-01-16T14:55:43.197397+0000","last_undegraded":"2024-01-16T14:55:43.197397+0000","last_fullsized":"2024-01-16T14:55:43.197397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:42:26.976555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192794+0000","last_change":"2024-01-16T14:55:43.185949+0000","last_active":"2024-01-16T14:55:43.192794+0000","last_peered":"2024-01-16T14:55:43.192794+0000","last_clean":"2024-01-16T14:55:43.192794+0000","last_became_active":"2024-01-16T14:55:43.185657+0000","last_became_peered":"2024-01-16T14:55:43.185657+0000","last_unstale":"2024-01-16T14:55:43.192794+0000","last_undegraded":"2024-01-16T14:55:43.192794+0000","last_fullsized":"2024-01-16T14:55:43.192794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:33:21.628613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193594+0000","last_change":"2024-01-16T14:55:43.188874+0000","last_active":"2024-01-16T14:55:43.193594+0000","last_peered":"2024-01-16T14:55:43.193594+0000","last_clean":"2024-01-16T14:55:43.193594+0000","last_became_active":"2024-01-16T14:55:43.188547+0000","last_became_peered":"2024-01-16T14:55:43.188547+0000","last_unstale":"2024-01-16T14:55:43.193594+0000","last_undegraded":"2024-01-16T14:55:43.193594+0000","last_fullsized":"2024-01-16T14:55:43.193594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:28.839366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195907+0000","last_change":"2024-01-16T14:55:43.192325+0000","last_active":"2024-01-16T14:55:43.195907+0000","last_peered":"2024-01-16T14:55:43.195907+0000","last_clean":"2024-01-16T14:55:43.195907+0000","last_became_active":"2024-01-16T14:55:43.191903+0000","last_became_peered":"2024-01-16T14:55:43.191903+0000","last_unstale":"2024-01-16T14:55:43.195907+0000","last_undegraded":"2024-01-16T14:55:43.195907+0000","last_fullsized":"2024-01-16T14:55:43.195907+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:01.991798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195294+0000","last_change":"2024-01-16T14:55:43.189139+0000","last_active":"2024-01-16T14:55:43.195294+0000","last_peered":"2024-01-16T14:55:43.195294+0000","last_clean":"2024-01-16T14:55:43.195294+0000","last_became_active":"2024-01-16T14:55:43.188826+0000","last_became_peered":"2024-01-16T14:55:43.188826+0000","last_unstale":"2024-01-16T14:55:43.195294+0000","last_undegraded":"2024-01-16T14:55:43.195294+0000","last_fullsized":"2024-01-16T14:55:43.195294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192723+0000","last_change":"2024-01-16T14:55:43.189390+0000","last_active":"2024-01-16T14:55:43.192723+0000","last_peered":"2024-01-16T14:55:43.192723+0000","last_clean":"2024-01-16T14:55:43.192723+0000","last_became_active":"2024-01-16T14:55:43.189085+0000","last_became_peered":"2024-01-16T14:55:43.189085+0000","last_unstale":"2024-01-16T14:55:43.192723+0000","last_undegraded":"2024-01-16T14:55:43.192723+0000","last_fullsized":"2024-01-16T14:55:43.192723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:54:20.701005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.190796+0000","last_change":"2024-01-16T14:55:43.179714+0000","last_active":"2024-01-16T14:55:43.190796+0000","last_peered":"2024-01-16T14:55:43.190796+0000","last_clean":"2024-01-16T14:55:43.190796+0000","last_became_active":"2024-01-16T14:55:43.179306+0000","last_became_peered":"2024-01-16T14:55:43.179306+0000","last_unstale":"2024-01-16T14:55:43.190796+0000","last_undegraded":"2024-01-16T14:55:43.190796+0000","last_fullsized":"2024-01-16T14:55:43.190796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:52.849751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197192+0000","last_change":"2024-01-16T14:55:43.194932+0000","last_active":"2024-01-16T14:55:43.197192+0000","last_peered":"2024-01-16T14:55:43.197192+0000","last_clean":"2024-01-16T14:55:43.197192+0000","last_became_active":"2024-01-16T14:55:43.194674+0000","last_became_peered":"2024-01-16T14:55:43.194674+0000","last_unstale":"2024-01-16T14:55:43.197192+0000","last_undegraded":"2024-01-16T14:55:43.197192+0000","last_fullsized":"2024-01-16T14:55:43.197192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:54:25.624931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192806+0000","last_change":"2024-01-16T14:55:43.185254+0000","last_active":"2024-01-16T14:55:43.192806+0000","last_peered":"2024-01-16T14:55:43.192806+0000","last_clean":"2024-01-16T14:55:43.192806+0000","last_became_active":"2024-01-16T14:55:43.184942+0000","last_became_peered":"2024-01-16T14:55:43.184942+0000","last_unstale":"2024-01-16T14:55:43.192806+0000","last_undegraded":"2024-01-16T14:55:43.192806+0000","last_fullsized":"2024-01-16T14:55:43.192806+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:59:47.307096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197174+0000","last_change":"2024-01-16T14:55:43.194349+0000","last_active":"2024-01-16T14:55:43.197174+0000","last_peered":"2024-01-16T14:55:43.197174+0000","last_clean":"2024-01-16T14:55:43.197174+0000","last_became_active":"2024-01-16T14:55:43.193956+0000","last_became_peered":"2024-01-16T14:55:43.193956+0000","last_unstale":"2024-01-16T14:55:43.197174+0000","last_undegraded":"2024-01-16T14:55:43.197174+0000","last_fullsized":"2024-01-16T14:55:43.197174+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198597+0000","last_change":"2024-01-16T14:55:43.193105+0000","last_active":"2024-01-16T14:55:43.198597+0000","last_peered":"2024-01-16T14:55:43.198597+0000","last_clean":"2024-01-16T14:55:43.198597+0000","last_became_active":"2024-01-16T14:55:43.192824+0000","last_became_peered":"2024-01-16T14:55:43.192824+0000","last_unstale":"2024-01-16T14:55:43.198597+0000","last_undegraded":"2024-01-16T14:55:43.198597+0000","last_fullsized":"2024-01-16T14:55:43.198597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196486+0000","last_change":"2024-01-16T14:55:43.191071+0000","last_active":"2024-01-16T14:55:43.196486+0000","last_peered":"2024-01-16T14:55:43.196486+0000","last_clean":"2024-01-16T14:55:43.196486+0000","last_became_active":"2024-01-16T14:55:43.190517+0000","last_became_peered":"2024-01-16T14:55:43.190517+0000","last_unstale":"2024-01-16T14:55:43.196486+0000","last_undegraded":"2024-01-16T14:55:43.196486+0000","last_fullsized":"2024-01-16T14:55:43.196486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:38.153820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196017+0000","last_change":"2024-01-16T14:55:43.188560+0000","last_active":"2024-01-16T14:55:43.196017+0000","last_peered":"2024-01-16T14:55:43.196017+0000","last_clean":"2024-01-16T14:55:43.196017+0000","last_became_active":"2024-01-16T14:55:43.188186+0000","last_became_peered":"2024-01-16T14:55:43.188186+0000","last_unstale":"2024-01-16T14:55:43.196017+0000","last_undegraded":"2024-01-16T14:55:43.196017+0000","last_fullsized":"2024-01-16T14:55:43.196017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:52.849751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.183357+0000","last_change":"2024-01-16T14:55:42.191801+0000","last_active":"2024-01-16T14:55:43.183357+0000","last_peered":"2024-01-16T14:55:43.183357+0000","last_clean":"2024-01-16T14:55:43.183357+0000","last_became_active":"2024-01-16T14:55:42.191512+0000","last_became_peered":"2024-01-16T14:55:42.191512+0000","last_unstale":"2024-01-16T14:55:43.183357+0000","last_undegraded":"2024-01-16T14:55:43.183357+0000","last_fullsized":"2024-01-16T14:55:43.183357+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:51.843541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196817+0000","last_change":"2024-01-16T14:55:43.189282+0000","last_active":"2024-01-16T14:55:43.196817+0000","last_peered":"2024-01-16T14:55:43.196817+0000","last_clean":"2024-01-16T14:55:43.196817+0000","last_became_active":"2024-01-16T14:55:43.189006+0000","last_became_peered":"2024-01-16T14:55:43.189006+0000","last_unstale":"2024-01-16T14:55:43.196817+0000","last_undegraded":"2024-01-16T14:55:43.196817+0000","last_fullsized":"2024-01-16T14:55:43.196817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:42.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186704+0000","last_change":"2024-01-16T14:55:42.190865+0000","last_active":"2024-01-16T14:55:43.186704+0000","last_peered":"2024-01-16T14:55:43.186704+0000","last_clean":"2024-01-16T14:55:43.186704+0000","last_became_active":"2024-01-16T14:55:42.190554+0000","last_became_peered":"2024-01-16T14:55:42.190554+0000","last_unstale":"2024-01-16T14:55:43.186704+0000","last_undegraded":"2024-01-16T14:55:43.186704+0000","last_fullsized":"2024-01-16T14:55:43.186704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192687+0000","last_change":"2024-01-16T14:55:43.184422+0000","last_active":"2024-01-16T14:55:43.192687+0000","last_peered":"2024-01-16T14:55:43.192687+0000","last_clean":"2024-01-16T14:55:43.192687+0000","last_became_active":"2024-01-16T14:55:43.184168+0000","last_became_peered":"2024-01-16T14:55:43.184168+0000","last_unstale":"2024-01-16T14:55:43.192687+0000","last_undegraded":"2024-01-16T14:55:43.192687+0000","last_fullsized":"2024-01-16T14:55:43.192687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T17:57:33.263332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175089+0000","last_change":"2024-01-16T14:55:42.191693+0000","last_active":"2024-01-16T14:55:43.175089+0000","last_peered":"2024-01-16T14:55:43.175089+0000","last_clean":"2024-01-16T14:55:43.175089+0000","last_became_active":"2024-01-16T14:55:42.191377+0000","last_became_peered":"2024-01-16T14:55:42.191377+0000","last_unstale":"2024-01-16T14:55:43.175089+0000","last_undegraded":"2024-01-16T14:55:43.175089+0000","last_fullsized":"2024-01-16T14:55:43.175089+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:20.715753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195431+0000","last_change":"2024-01-16T14:55:43.184609+0000","last_active":"2024-01-16T14:55:43.195431+0000","last_peered":"2024-01-16T14:55:43.195431+0000","last_clean":"2024-01-16T14:55:43.195431+0000","last_became_active":"2024-01-16T14:55:43.184325+0000","last_became_peered":"2024-01-16T14:55:43.184325+0000","last_unstale":"2024-01-16T14:55:43.195431+0000","last_undegraded":"2024-01-16T14:55:43.195431+0000","last_fullsized":"2024-01-16T14:55:43.195431+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186761+0000","last_change":"2024-01-16T14:55:42.189119+0000","last_active":"2024-01-16T14:55:43.186761+0000","last_peered":"2024-01-16T14:55:43.186761+0000","last_clean":"2024-01-16T14:55:43.186761+0000","last_became_active":"2024-01-16T14:55:42.188886+0000","last_became_peered":"2024-01-16T14:55:42.188886+0000","last_unstale":"2024-01-16T14:55:43.186761+0000","last_undegraded":"2024-01-16T14:55:43.186761+0000","last_fullsized":"2024-01-16T14:55:43.186761+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:59:46.300886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192446+0000","last_change":"2024-01-16T14:55:43.188240+0000","last_active":"2024-01-16T14:55:43.192446+0000","last_peered":"2024-01-16T14:55:43.192446+0000","last_clean":"2024-01-16T14:55:43.192446+0000","last_became_active":"2024-01-16T14:55:43.187884+0000","last_became_peered":"2024-01-16T14:55:43.187884+0000","last_unstale":"2024-01-16T14:55:43.192446+0000","last_undegraded":"2024-01-16T14:55:43.192446+0000","last_fullsized":"2024-01-16T14:55:43.192446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176660+0000","last_change":"2024-01-16T14:55:42.182853+0000","last_active":"2024-01-16T14:55:43.176660+0000","last_peered":"2024-01-16T14:55:43.176660+0000","last_clean":"2024-01-16T14:55:43.176660+0000","last_became_active":"2024-01-16T14:55:42.182520+0000","last_became_peered":"2024-01-16T14:55:42.182520+0000","last_unstale":"2024-01-16T14:55:43.176660+0000","last_undegraded":"2024-01-16T14:55:43.176660+0000","last_fullsized":"2024-01-16T14:55:43.176660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:37.147610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201102+0000","last_change":"2024-01-16T14:55:43.197108+0000","last_active":"2024-01-16T14:55:43.201102+0000","last_peered":"2024-01-16T14:55:43.201102+0000","last_clean":"2024-01-16T14:55:43.201102+0000","last_became_active":"2024-01-16T14:55:43.196813+0000","last_became_peered":"2024-01-16T14:55:43.196813+0000","last_unstale":"2024-01-16T14:55:43.201102+0000","last_undegraded":"2024-01-16T14:55:43.201102+0000","last_fullsized":"2024-01-16T14:55:43.201102+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:59.200645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176394+0000","last_change":"2024-01-16T14:55:42.183013+0000","last_active":"2024-01-16T14:55:43.176394+0000","last_peered":"2024-01-16T14:55:43.176394+0000","last_clean":"2024-01-16T14:55:43.176394+0000","last_became_active":"2024-01-16T14:55:42.182703+0000","last_became_peered":"2024-01-16T14:55:42.182703+0000","last_unstale":"2024-01-16T14:55:43.176394+0000","last_undegraded":"2024-01-16T14:55:43.176394+0000","last_fullsized":"2024-01-16T14:55:43.176394+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:27.833157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.200867+0000","last_change":"2024-01-16T14:55:43.193837+0000","last_active":"2024-01-16T14:55:43.200867+0000","last_peered":"2024-01-16T14:55:43.200867+0000","last_clean":"2024-01-16T14:55:43.200867+0000","last_became_active":"2024-01-16T14:55:43.193563+0000","last_became_peered":"2024-01-16T14:55:43.193563+0000","last_unstale":"2024-01-16T14:55:43.200867+0000","last_undegraded":"2024-01-16T14:55:43.200867+0000","last_fullsized":"2024-01-16T14:55:43.200867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:57:21.003387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173463+0000","last_change":"2024-01-16T14:55:42.189207+0000","last_active":"2024-01-16T14:55:43.173463+0000","last_peered":"2024-01-16T14:55:43.173463+0000","last_clean":"2024-01-16T14:55:43.173463+0000","last_became_active":"2024-01-16T14:55:42.188884+0000","last_became_peered":"2024-01-16T14:55:42.188884+0000","last_unstale":"2024-01-16T14:55:43.173463+0000","last_undegraded":"2024-01-16T14:55:43.173463+0000","last_fullsized":"2024-01-16T14:55:43.173463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196212+0000","last_change":"2024-01-16T14:55:43.190160+0000","last_active":"2024-01-16T14:55:43.196212+0000","last_peered":"2024-01-16T14:55:43.196212+0000","last_clean":"2024-01-16T14:55:43.196212+0000","last_became_active":"2024-01-16T14:55:43.189580+0000","last_became_peered":"2024-01-16T14:55:43.189580+0000","last_unstale":"2024-01-16T14:55:43.196212+0000","last_undegraded":"2024-01-16T14:55:43.196212+0000","last_fullsized":"2024-01-16T14:55:43.196212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:21.027877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175851+0000","last_change":"2024-01-16T14:55:42.186227+0000","last_active":"2024-01-16T14:55:43.175851+0000","last_peered":"2024-01-16T14:55:43.175851+0000","last_clean":"2024-01-16T14:55:43.175851+0000","last_became_active":"2024-01-16T14:55:42.185936+0000","last_became_peered":"2024-01-16T14:55:42.185936+0000","last_unstale":"2024-01-16T14:55:43.175851+0000","last_undegraded":"2024-01-16T14:55:43.175851+0000","last_fullsized":"2024-01-16T14:55:43.175851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:09:15.337306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196849+0000","last_change":"2024-01-16T14:55:43.194724+0000","last_active":"2024-01-16T14:55:43.196849+0000","last_peered":"2024-01-16T14:55:43.196849+0000","last_clean":"2024-01-16T14:55:43.196849+0000","last_became_active":"2024-01-16T14:55:43.194312+0000","last_became_peered":"2024-01-16T14:55:43.194312+0000","last_unstale":"2024-01-16T14:55:43.196849+0000","last_undegraded":"2024-01-16T14:55:43.196849+0000","last_fullsized":"2024-01-16T14:55:43.196849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174257+0000","last_change":"2024-01-16T14:55:42.186300+0000","last_active":"2024-01-16T14:55:43.174257+0000","last_peered":"2024-01-16T14:55:43.174257+0000","last_clean":"2024-01-16T14:55:43.174257+0000","last_became_active":"2024-01-16T14:55:42.186016+0000","last_became_peered":"2024-01-16T14:55:42.186016+0000","last_unstale":"2024-01-16T14:55:43.174257+0000","last_undegraded":"2024-01-16T14:55:43.174257+0000","last_fullsized":"2024-01-16T14:55:43.174257+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:58.194435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187327+0000","last_change":"2024-01-16T14:55:43.180280+0000","last_active":"2024-01-16T14:55:43.187327+0000","last_peered":"2024-01-16T14:55:43.187327+0000","last_clean":"2024-01-16T14:55:43.187327+0000","last_became_active":"2024-01-16T14:55:43.179960+0000","last_became_peered":"2024-01-16T14:55:43.179960+0000","last_unstale":"2024-01-16T14:55:43.187327+0000","last_undegraded":"2024-01-16T14:55:43.187327+0000","last_fullsized":"2024-01-16T14:55:43.187327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.186912+0000","last_change":"2024-01-16T14:55:42.189901+0000","last_active":"2024-01-16T14:55:43.186912+0000","last_peered":"2024-01-16T14:55:43.186912+0000","last_clean":"2024-01-16T14:55:43.186912+0000","last_became_active":"2024-01-16T14:55:42.189275+0000","last_became_peered":"2024-01-16T14:55:42.189275+0000","last_unstale":"2024-01-16T14:55:43.186912+0000","last_undegraded":"2024-01-16T14:55:43.186912+0000","last_fullsized":"2024-01-16T14:55:43.186912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192642+0000","last_change":"2024-01-16T14:55:43.189681+0000","last_active":"2024-01-16T14:55:43.192642+0000","last_peered":"2024-01-16T14:55:43.192642+0000","last_clean":"2024-01-16T14:55:43.192642+0000","last_became_active":"2024-01-16T14:55:43.189390+0000","last_became_peered":"2024-01-16T14:55:43.189390+0000","last_unstale":"2024-01-16T14:55:43.192642+0000","last_undegraded":"2024-01-16T14:55:43.192642+0000","last_fullsized":"2024-01-16T14:55:43.192642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:30.973623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188821+0000","last_change":"2024-01-16T14:55:42.189180+0000","last_active":"2024-01-16T14:55:43.188821+0000","last_peered":"2024-01-16T14:55:43.188821+0000","last_clean":"2024-01-16T14:55:43.188821+0000","last_became_active":"2024-01-16T14:55:42.188906+0000","last_became_peered":"2024-01-16T14:55:42.188906+0000","last_unstale":"2024-01-16T14:55:43.188821+0000","last_undegraded":"2024-01-16T14:55:43.188821+0000","last_fullsized":"2024-01-16T14:55:43.188821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:23:03.103944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201193+0000","last_change":"2024-01-16T14:55:43.198186+0000","last_active":"2024-01-16T14:55:43.201193+0000","last_peered":"2024-01-16T14:55:43.201193+0000","last_clean":"2024-01-16T14:55:43.201193+0000","last_became_active":"2024-01-16T14:55:43.197930+0000","last_became_peered":"2024-01-16T14:55:43.197930+0000","last_unstale":"2024-01-16T14:55:43.201193+0000","last_undegraded":"2024-01-16T14:55:43.201193+0000","last_fullsized":"2024-01-16T14:55:43.201193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'5","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174517+0000","last_change":"2024-01-16T14:55:42.188042+0000","last_active":"2024-01-16T14:55:43.174517+0000","last_peered":"2024-01-16T14:55:43.174517+0000","last_clean":"2024-01-16T14:55:43.174517+0000","last_became_active":"2024-01-16T14:55:42.187726+0000","last_became_peered":"2024-01-16T14:55:42.187726+0000","last_unstale":"2024-01-16T14:55:43.174517+0000","last_undegraded":"2024-01-16T14:55:43.174517+0000","last_fullsized":"2024-01-16T14:55:43.174517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:28:00.985588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197466+0000","last_change":"2024-01-16T14:55:43.194584+0000","last_active":"2024-01-16T14:55:43.197466+0000","last_peered":"2024-01-16T14:55:43.197466+0000","last_clean":"2024-01-16T14:55:43.197466+0000","last_became_active":"2024-01-16T14:55:43.194325+0000","last_became_peered":"2024-01-16T14:55:43.194325+0000","last_unstale":"2024-01-16T14:55:43.197466+0000","last_undegraded":"2024-01-16T14:55:43.197466+0000","last_fullsized":"2024-01-16T14:55:43.197466+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:59.897489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176698+0000","last_change":"2024-01-16T14:55:42.186365+0000","last_active":"2024-01-16T14:55:43.176698+0000","last_peered":"2024-01-16T14:55:43.176698+0000","last_clean":"2024-01-16T14:55:43.176698+0000","last_became_active":"2024-01-16T14:55:42.186065+0000","last_became_peered":"2024-01-16T14:55:42.186065+0000","last_unstale":"2024-01-16T14:55:43.176698+0000","last_undegraded":"2024-01-16T14:55:43.176698+0000","last_fullsized":"2024-01-16T14:55:43.176698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192680+0000","last_change":"2024-01-16T14:55:43.189567+0000","last_active":"2024-01-16T14:55:43.192680+0000","last_peered":"2024-01-16T14:55:43.192680+0000","last_clean":"2024-01-16T14:55:43.192680+0000","last_became_active":"2024-01-16T14:55:43.189233+0000","last_became_peered":"2024-01-16T14:55:43.189233+0000","last_unstale":"2024-01-16T14:55:43.192680+0000","last_undegraded":"2024-01-16T14:55:43.192680+0000","last_fullsized":"2024-01-16T14:55:43.192680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:54:25.624931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173878+0000","last_change":"2024-01-16T14:55:42.196199+0000","last_active":"2024-01-16T14:55:43.173878+0000","last_peered":"2024-01-16T14:55:43.173878+0000","last_clean":"2024-01-16T14:55:43.173878+0000","last_became_active":"2024-01-16T14:55:42.195882+0000","last_became_peered":"2024-01-16T14:55:42.195882+0000","last_unstale":"2024-01-16T14:55:43.173878+0000","last_undegraded":"2024-01-16T14:55:43.173878+0000","last_fullsized":"2024-01-16T14:55:43.173878+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:06:58.194435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196443+0000","last_change":"2024-01-16T14:55:43.190160+0000","last_active":"2024-01-16T14:55:43.196443+0000","last_peered":"2024-01-16T14:55:43.196443+0000","last_clean":"2024-01-16T14:55:43.196443+0000","last_became_active":"2024-01-16T14:55:43.189576+0000","last_became_peered":"2024-01-16T14:55:43.189576+0000","last_unstale":"2024-01-16T14:55:43.196443+0000","last_undegraded":"2024-01-16T14:55:43.196443+0000","last_fullsized":"2024-01-16T14:55:43.196443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173557+0000","last_change":"2024-01-16T14:55:42.186853+0000","last_active":"2024-01-16T14:55:43.173557+0000","last_peered":"2024-01-16T14:55:43.173557+0000","last_clean":"2024-01-16T14:55:43.173557+0000","last_became_active":"2024-01-16T14:55:42.186564+0000","last_became_peered":"2024-01-16T14:55:42.186564+0000","last_unstale":"2024-01-16T14:55:43.173557+0000","last_undegraded":"2024-01-16T14:55:43.173557+0000","last_fullsized":"2024-01-16T14:55:43.173557+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198621+0000","last_change":"2024-01-16T14:55:43.193616+0000","last_active":"2024-01-16T14:55:43.198621+0000","last_peered":"2024-01-16T14:55:43.198621+0000","last_clean":"2024-01-16T14:55:43.198621+0000","last_became_active":"2024-01-16T14:55:43.193357+0000","last_became_peered":"2024-01-16T14:55:43.193357+0000","last_unstale":"2024-01-16T14:55:43.198621+0000","last_undegraded":"2024-01-16T14:55:43.198621+0000","last_fullsized":"2024-01-16T14:55:43.198621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:18.080820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175588+0000","last_change":"2024-01-16T14:55:42.185458+0000","last_active":"2024-01-16T14:55:43.175588+0000","last_peered":"2024-01-16T14:55:43.175588+0000","last_clean":"2024-01-16T14:55:43.175588+0000","last_became_active":"2024-01-16T14:55:42.185109+0000","last_became_peered":"2024-01-16T14:55:42.185109+0000","last_unstale":"2024-01-16T14:55:43.175588+0000","last_undegraded":"2024-01-16T14:55:43.175588+0000","last_fullsized":"2024-01-16T14:55:43.175588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:23:03.103944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.200855+0000","last_change":"2024-01-16T14:55:43.192503+0000","last_active":"2024-01-16T14:55:43.200855+0000","last_peered":"2024-01-16T14:55:43.200855+0000","last_clean":"2024-01-16T14:55:43.200855+0000","last_became_active":"2024-01-16T14:55:43.192243+0000","last_became_peered":"2024-01-16T14:55:43.192243+0000","last_unstale":"2024-01-16T14:55:43.200855+0000","last_undegraded":"2024-01-16T14:55:43.200855+0000","last_fullsized":"2024-01-16T14:55:43.200855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:37:51.884099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173642+0000","last_change":"2024-01-16T14:55:42.186447+0000","last_active":"2024-01-16T14:55:43.173642+0000","last_peered":"2024-01-16T14:55:43.173642+0000","last_clean":"2024-01-16T14:55:43.173642+0000","last_became_active":"2024-01-16T14:55:42.186161+0000","last_became_peered":"2024-01-16T14:55:42.186161+0000","last_unstale":"2024-01-16T14:55:43.173642+0000","last_undegraded":"2024-01-16T14:55:43.173642+0000","last_fullsized":"2024-01-16T14:55:43.173642+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201146+0000","last_change":"2024-01-16T14:55:43.197986+0000","last_active":"2024-01-16T14:55:43.201146+0000","last_peered":"2024-01-16T14:55:43.201146+0000","last_clean":"2024-01-16T14:55:43.201146+0000","last_became_active":"2024-01-16T14:55:43.197600+0000","last_became_peered":"2024-01-16T14:55:43.197600+0000","last_unstale":"2024-01-16T14:55:43.201146+0000","last_undegraded":"2024-01-16T14:55:43.201146+0000","last_fullsized":"2024-01-16T14:55:43.201146+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:25:11.688854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.177076+0000","last_change":"2024-01-16T14:55:42.191548+0000","last_active":"2024-01-16T14:55:43.177076+0000","last_peered":"2024-01-16T14:55:43.177076+0000","last_clean":"2024-01-16T14:55:43.177076+0000","last_became_active":"2024-01-16T14:55:42.191160+0000","last_became_peered":"2024-01-16T14:55:42.191160+0000","last_unstale":"2024-01-16T14:55:43.177076+0000","last_undegraded":"2024-01-16T14:55:43.177076+0000","last_fullsized":"2024-01-16T14:55:43.177076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:50:58.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195248+0000","last_change":"2024-01-16T14:55:43.182487+0000","last_active":"2024-01-16T14:55:43.195248+0000","last_peered":"2024-01-16T14:55:43.195248+0000","last_clean":"2024-01-16T14:55:43.195248+0000","last_became_active":"2024-01-16T14:55:43.182152+0000","last_became_peered":"2024-01-16T14:55:43.182152+0000","last_unstale":"2024-01-16T14:55:43.195248+0000","last_undegraded":"2024-01-16T14:55:43.195248+0000","last_fullsized":"2024-01-16T14:55:43.195248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:23:50.310371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.177184+0000","last_change":"2024-01-16T14:55:42.189643+0000","last_active":"2024-01-16T14:55:43.177184+0000","last_peered":"2024-01-16T14:55:43.177184+0000","last_clean":"2024-01-16T14:55:43.177184+0000","last_became_active":"2024-01-16T14:55:42.189344+0000","last_became_peered":"2024-01-16T14:55:42.189344+0000","last_unstale":"2024-01-16T14:55:43.177184+0000","last_undegraded":"2024-01-16T14:55:43.177184+0000","last_fullsized":"2024-01-16T14:55:43.177184+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:05.215220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.193489+0000","last_change":"2024-01-16T14:55:43.190839+0000","last_active":"2024-01-16T14:55:43.193489+0000","last_peered":"2024-01-16T14:55:43.193489+0000","last_clean":"2024-01-16T14:55:43.193489+0000","last_became_active":"2024-01-16T14:55:43.190493+0000","last_became_peered":"2024-01-16T14:55:43.190493+0000","last_unstale":"2024-01-16T14:55:43.193489+0000","last_undegraded":"2024-01-16T14:55:43.193489+0000","last_fullsized":"2024-01-16T14:55:43.193489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:04:32.590914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176514+0000","last_change":"2024-01-16T14:55:42.183357+0000","last_active":"2024-01-16T14:55:43.176514+0000","last_peered":"2024-01-16T14:55:43.176514+0000","last_clean":"2024-01-16T14:55:43.176514+0000","last_became_active":"2024-01-16T14:55:42.183015+0000","last_became_peered":"2024-01-16T14:55:42.183015+0000","last_unstale":"2024-01-16T14:55:43.176514+0000","last_undegraded":"2024-01-16T14:55:43.176514+0000","last_fullsized":"2024-01-16T14:55:43.176514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:28:51.843541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195946+0000","last_change":"2024-01-16T14:55:43.193474+0000","last_active":"2024-01-16T14:55:43.195946+0000","last_peered":"2024-01-16T14:55:43.195946+0000","last_clean":"2024-01-16T14:55:43.195946+0000","last_became_active":"2024-01-16T14:55:43.193179+0000","last_became_peered":"2024-01-16T14:55:43.193179+0000","last_unstale":"2024-01-16T14:55:43.195946+0000","last_undegraded":"2024-01-16T14:55:43.195946+0000","last_fullsized":"2024-01-16T14:55:43.195946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:25:12.024242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182085+0000","last_change":"2024-01-16T14:55:42.193921+0000","last_active":"2024-01-16T14:55:43.182085+0000","last_peered":"2024-01-16T14:55:43.182085+0000","last_clean":"2024-01-16T14:55:43.182085+0000","last_became_active":"2024-01-16T14:55:42.193454+0000","last_became_peered":"2024-01-16T14:55:42.193454+0000","last_unstale":"2024-01-16T14:55:43.182085+0000","last_undegraded":"2024-01-16T14:55:43.182085+0000","last_fullsized":"2024-01-16T14:55:43.182085+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T18:51:37.147610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196827+0000","last_change":"2024-01-16T14:55:43.191599+0000","last_active":"2024-01-16T14:55:43.196827+0000","last_peered":"2024-01-16T14:55:43.196827+0000","last_clean":"2024-01-16T14:55:43.196827+0000","last_became_active":"2024-01-16T14:55:43.191229+0000","last_became_peered":"2024-01-16T14:55:43.191229+0000","last_unstale":"2024-01-16T14:55:43.196827+0000","last_undegraded":"2024-01-16T14:55:43.196827+0000","last_fullsized":"2024-01-16T14:55:43.196827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:26:17.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182847+0000","last_change":"2024-01-16T14:55:42.196610+0000","last_active":"2024-01-16T14:55:43.182847+0000","last_peered":"2024-01-16T14:55:43.182847+0000","last_clean":"2024-01-16T14:55:43.182847+0000","last_became_active":"2024-01-16T14:55:42.196315+0000","last_became_peered":"2024-01-16T14:55:42.196315+0000","last_unstale":"2024-01-16T14:55:43.182847+0000","last_undegraded":"2024-01-16T14:55:43.182847+0000","last_fullsized":"2024-01-16T14:55:43.182847+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:20.021667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196387+0000","last_change":"2024-01-16T14:55:43.194688+0000","last_active":"2024-01-16T14:55:43.196387+0000","last_peered":"2024-01-16T14:55:43.196387+0000","last_clean":"2024-01-16T14:55:43.196387+0000","last_became_active":"2024-01-16T14:55:43.193736+0000","last_became_peered":"2024-01-16T14:55:43.193736+0000","last_unstale":"2024-01-16T14:55:43.196387+0000","last_undegraded":"2024-01-16T14:55:43.196387+0000","last_fullsized":"2024-01-16T14:55:43.196387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:32:59.945357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176196+0000","last_change":"2024-01-16T14:55:42.196613+0000","last_active":"2024-01-16T14:55:43.176196+0000","last_peered":"2024-01-16T14:55:43.176196+0000","last_clean":"2024-01-16T14:55:43.176196+0000","last_became_active":"2024-01-16T14:55:42.196342+0000","last_became_peered":"2024-01-16T14:55:42.196342+0000","last_unstale":"2024-01-16T14:55:43.176196+0000","last_undegraded":"2024-01-16T14:55:43.176196+0000","last_fullsized":"2024-01-16T14:55:43.176196+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:20.021667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.197103+0000","last_change":"2024-01-16T14:55:43.193907+0000","last_active":"2024-01-16T14:55:43.197103+0000","last_peered":"2024-01-16T14:55:43.197103+0000","last_clean":"2024-01-16T14:55:43.197103+0000","last_became_active":"2024-01-16T14:55:43.193541+0000","last_became_peered":"2024-01-16T14:55:43.193541+0000","last_unstale":"2024-01-16T14:55:43.197103+0000","last_undegraded":"2024-01-16T14:55:43.197103+0000","last_fullsized":"2024-01-16T14:55:43.197103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:21.721962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.188113+0000","last_change":"2024-01-16T14:55:42.189118+0000","last_active":"2024-01-16T14:55:43.188113+0000","last_peered":"2024-01-16T14:55:43.188113+0000","last_clean":"2024-01-16T14:55:43.188113+0000","last_became_active":"2024-01-16T14:55:42.188821+0000","last_became_peered":"2024-01-16T14:55:42.188821+0000","last_unstale":"2024-01-16T14:55:43.188113+0000","last_undegraded":"2024-01-16T14:55:43.188113+0000","last_fullsized":"2024-01-16T14:55:43.188113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T02:06:08.814677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.195034+0000","last_change":"2024-01-16T14:55:43.192878+0000","last_active":"2024-01-16T14:55:43.195034+0000","last_peered":"2024-01-16T14:55:43.195034+0000","last_clean":"2024-01-16T14:55:43.195034+0000","last_became_active":"2024-01-16T14:55:43.192545+0000","last_became_peered":"2024-01-16T14:55:43.192545+0000","last_unstale":"2024-01-16T14:55:43.195034+0000","last_undegraded":"2024-01-16T14:55:43.195034+0000","last_fullsized":"2024-01-16T14:55:43.195034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T19:48:30.261767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173396+0000","last_change":"2024-01-16T14:55:42.199427+0000","last_active":"2024-01-16T14:55:43.173396+0000","last_peered":"2024-01-16T14:55:43.173396+0000","last_clean":"2024-01-16T14:55:43.173396+0000","last_became_active":"2024-01-16T14:55:42.199117+0000","last_became_peered":"2024-01-16T14:55:42.199117+0000","last_unstale":"2024-01-16T14:55:43.173396+0000","last_undegraded":"2024-01-16T14:55:43.173396+0000","last_fullsized":"2024-01-16T14:55:43.173396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:41:27.833157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.201199+0000","last_change":"2024-01-16T14:55:43.198245+0000","last_active":"2024-01-16T14:55:43.201199+0000","last_peered":"2024-01-16T14:55:43.201199+0000","last_clean":"2024-01-16T14:55:43.201199+0000","last_became_active":"2024-01-16T14:55:43.197959+0000","last_became_peered":"2024-01-16T14:55:43.197959+0000","last_unstale":"2024-01-16T14:55:43.201199+0000","last_undegraded":"2024-01-16T14:55:43.201199+0000","last_fullsized":"2024-01-16T14:55:43.201199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:21.721962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187066+0000","last_change":"2024-01-16T14:55:42.189680+0000","last_active":"2024-01-16T14:55:43.187066+0000","last_peered":"2024-01-16T14:55:43.187066+0000","last_clean":"2024-01-16T14:55:43.187066+0000","last_became_active":"2024-01-16T14:55:42.188880+0000","last_became_peered":"2024-01-16T14:55:42.188880+0000","last_unstale":"2024-01-16T14:55:43.187066+0000","last_undegraded":"2024-01-16T14:55:43.187066+0000","last_fullsized":"2024-01-16T14:55:43.187066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:29.967413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198747+0000","last_change":"2024-01-16T14:55:43.197141+0000","last_active":"2024-01-16T14:55:43.198747+0000","last_peered":"2024-01-16T14:55:43.198747+0000","last_clean":"2024-01-16T14:55:43.198747+0000","last_became_active":"2024-01-16T14:55:43.196841+0000","last_became_peered":"2024-01-16T14:55:43.196841+0000","last_unstale":"2024-01-16T14:55:43.198747+0000","last_undegraded":"2024-01-16T14:55:43.198747+0000","last_fullsized":"2024-01-16T14:55:43.198747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:42.105275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.174957+0000","last_change":"2024-01-16T14:55:42.192877+0000","last_active":"2024-01-16T14:55:43.174957+0000","last_peered":"2024-01-16T14:55:43.174957+0000","last_clean":"2024-01-16T14:55:43.174957+0000","last_became_active":"2024-01-16T14:55:42.192524+0000","last_became_peered":"2024-01-16T14:55:42.192524+0000","last_unstale":"2024-01-16T14:55:43.174957+0000","last_undegraded":"2024-01-16T14:55:43.174957+0000","last_fullsized":"2024-01-16T14:55:43.174957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:13:20.715753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196875+0000","last_change":"2024-01-16T14:55:43.190880+0000","last_active":"2024-01-16T14:55:43.196875+0000","last_peered":"2024-01-16T14:55:43.196875+0000","last_clean":"2024-01-16T14:55:43.196875+0000","last_became_active":"2024-01-16T14:55:43.190600+0000","last_became_peered":"2024-01-16T14:55:43.190600+0000","last_unstale":"2024-01-16T14:55:43.196875+0000","last_undegraded":"2024-01-16T14:55:43.196875+0000","last_fullsized":"2024-01-16T14:55:43.196875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:09:16.343516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182181+0000","last_change":"2024-01-16T14:55:42.194306+0000","last_active":"2024-01-16T14:55:43.182181+0000","last_peered":"2024-01-16T14:55:43.182181+0000","last_clean":"2024-01-16T14:55:43.182181+0000","last_became_active":"2024-01-16T14:55:42.193919+0000","last_became_peered":"2024-01-16T14:55:42.193919+0000","last_unstale":"2024-01-16T14:55:43.182181+0000","last_undegraded":"2024-01-16T14:55:43.182181+0000","last_fullsized":"2024-01-16T14:55:43.182181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:21:17.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.198239+0000","last_change":"2024-01-16T14:55:43.195458+0000","last_active":"2024-01-16T14:55:43.198239+0000","last_peered":"2024-01-16T14:55:43.198239+0000","last_clean":"2024-01-16T14:55:43.198239+0000","last_became_active":"2024-01-16T14:55:43.194941+0000","last_became_peered":"2024-01-16T14:55:43.194941+0000","last_unstale":"2024-01-16T14:55:43.198239+0000","last_undegraded":"2024-01-16T14:55:43.198239+0000","last_fullsized":"2024-01-16T14:55:43.198239+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T23:30:21.027877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.173672+0000","last_change":"2024-01-16T14:55:42.201893+0000","last_active":"2024-01-16T14:55:43.173672+0000","last_peered":"2024-01-16T14:55:43.173672+0000","last_clean":"2024-01-16T14:55:43.173672+0000","last_became_active":"2024-01-16T14:55:42.201563+0000","last_became_peered":"2024-01-16T14:55:42.201563+0000","last_unstale":"2024-01-16T14:55:43.173672+0000","last_undegraded":"2024-01-16T14:55:43.173672+0000","last_fullsized":"2024-01-16T14:55:43.173672+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-18T00:17:29.967413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.187319+0000","last_change":"2024-01-16T14:55:43.180136+0000","last_active":"2024-01-16T14:55:43.187319+0000","last_peered":"2024-01-16T14:55:43.187319+0000","last_clean":"2024-01-16T14:55:43.187319+0000","last_became_active":"2024-01-16T14:55:43.179819+0000","last_became_peered":"2024-01-16T14:55:43.179819+0000","last_unstale":"2024-01-16T14:55:43.187319+0000","last_undegraded":"2024-01-16T14:55:43.187319+0000","last_fullsized":"2024-01-16T14:55:43.187319+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T20:14:13.682083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.176182+0000","last_change":"2024-01-16T14:55:42.186333+0000","last_active":"2024-01-16T14:55:43.176182+0000","last_peered":"2024-01-16T14:55:43.176182+0000","last_clean":"2024-01-16T14:55:43.176182+0000","last_became_active":"2024-01-16T14:55:42.186037+0000","last_became_peered":"2024-01-16T14:55:42.186037+0000","last_unstale":"2024-01-16T14:55:43.176182+0000","last_undegraded":"2024-01-16T14:55:43.176182+0000","last_fullsized":"2024-01-16T14:55:43.176182+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:16:41.099065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.192763+0000","last_change":"2024-01-16T14:55:43.185473+0000","last_active":"2024-01-16T14:55:43.192763+0000","last_peered":"2024-01-16T14:55:43.192763+0000","last_clean":"2024-01-16T14:55:43.192763+0000","last_became_active":"2024-01-16T14:55:43.185167+0000","last_became_peered":"2024-01-16T14:55:43.185167+0000","last_unstale":"2024-01-16T14:55:43.192763+0000","last_undegraded":"2024-01-16T14:55:43.192763+0000","last_fullsized":"2024-01-16T14:55:43.192763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T16:24:53.534604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.175764+0000","last_change":"2024-01-16T14:55:42.188428+0000","last_active":"2024-01-16T14:55:43.175764+0000","last_peered":"2024-01-16T14:55:43.175764+0000","last_clean":"2024-01-16T14:55:43.175764+0000","last_became_active":"2024-01-16T14:55:42.188123+0000","last_became_peered":"2024-01-16T14:55:42.188123+0000","last_unstale":"2024-01-16T14:55:43.175764+0000","last_undegraded":"2024-01-16T14:55:43.175764+0000","last_fullsized":"2024-01-16T14:55:43.175764+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T15:59:15.420678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.196467+0000","last_change":"2024-01-16T14:55:43.191677+0000","last_active":"2024-01-16T14:55:43.196467+0000","last_peered":"2024-01-16T14:55:43.196467+0000","last_clean":"2024-01-16T14:55:43.196467+0000","last_became_active":"2024-01-16T14:55:43.191385+0000","last_became_peered":"2024-01-16T14:55:43.191385+0000","last_unstale":"2024-01-16T14:55:43.196467+0000","last_undegraded":"2024-01-16T14:55:43.196467+0000","last_fullsized":"2024-01-16T14:55:43.196467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:42.162843+0000","last_clean_scrub_stamp":"2024-01-16T14:55:42.162843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:28:06.221430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-01-16T14:55:43.182150+0000","last_change":"2024-01-16T14:55:42.191276+0000","last_active":"2024-01-16T14:55:43.182150+0000","last_peered":"2024-01-16T14:55:43.182150+0000","last_clean":"2024-01-16T14:55:43.182150+0000","last_became_active":"2024-01-16T14:55:42.190987+0000","last_became_peered":"2024-01-16T14:55:42.190987+0000","last_unstale":"2024-01-16T14:55:43.182150+0000","last_undegraded":"2024-01-16T14:55:43.182150+0000","last_fullsized":"2024-01-16T14:55:43.182150+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:55:41.156633+0000","last_clean_scrub_stamp":"2024-01-16T14:55:41.156633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T22:25:11.114865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-01-16T14:55:42.171320+0000","last_change":"2024-01-16T14:54:41.352587+0000","last_active":"2024-01-16T14:55:42.171320+0000","last_peered":"2024-01-16T14:55:42.171320+0000","last_clean":"2024-01-16T14:55:42.171320+0000","last_became_active":"2024-01-16T14:54:41.351427+0000","last_became_peered":"2024-01-16T14:54:41.351427+0000","last_unstale":"2024-01-16T14:55:42.171320+0000","last_undegraded":"2024-01-16T14:55:42.171320+0000","last_fullsized":"2024-01-16T14:55:42.171320+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-16T14:54:39.808058+0000","last_clean_scrub_stamp":"2024-01-16T14:54:39.808058+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T21:08:24.012912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":762,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":716382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57399999999999995}]},{"osd":1,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.70899999999999996}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.755}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.625}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":126014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53100000000000003}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51500000000000001}]},{"osd":3,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":132170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":0.44},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.45600000000000002}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.55000000000000004}]},{"osd":7,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.496},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":925696,"data_stored":715763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.313},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.5}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59699999999999998}]},{"osd":6,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.629},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":125395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":1.1459999999999999}]},{"osd":1,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":2.1349999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":2.2759999999999998}]},{"osd":3,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":1.008},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":1.2110000000000001}]},{"osd":4,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.5},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.96599999999999997}]},{"osd":7,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.93400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":132170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35999999999999999}]},{"osd":1,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.315},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.40200000000000002}]},{"osd":4,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.63},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.57299999999999995}]},{"osd":5,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.47399999999999998}]},{"osd":6,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Jan 16 14:55:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.54100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67400000000000004}]},{"osd":2,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53400000000000003}]},{"osd":3,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66100000000000003}]},{"osd":6,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Jan 16 14:55:39 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.7850000000000001,"5min":2.7850000000000001,"15min":2.7850000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68700000000000006}]},{"osd":3,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Jan 16 14:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-16T14:55:48.057 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-16T14:55:48.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-16T14:55:48.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.257+0000 7f4b12bc7700 1 Processor -- start 2024-01-16T14:55:48.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.257+0000 7f4b12bc7700 1 -- start start 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b12bc7700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b0c1478c0 0x7f4b0c0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b12bc7700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 0x7f4b0c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b12bc7700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c06dbe0 con 0x7f4b0c161a90 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b12bc7700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0c147ca0 con 0x7f4b0c1478c0 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b12bc7700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0c1622e0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b10963700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b0c1478c0 0x7f4b0c0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b0bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b0c161a90 0x7f4b0c06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43952/0 (socket says 172.21.15.26:43952) 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b0bfff700 1 -- 172.21.15.26:0/2253282447 learned_addr learned my addr 172.21.15.26:0/2253282447 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b11164700 1 --2- 172.21.15.26:0/2253282447 >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 0x7f4b0c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b10963700 1 -- 172.21.15.26:0/2253282447 >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 msgr2=0x7f4b0c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b10963700 1 --2- 172.21.15.26:0/2253282447 >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 0x7f4b0c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.258+0000 7f4b10963700 1 -- 172.21.15.26:0/2253282447 >> v1:172.21.15.26:6789/0 conn(0x7f4b0c161a90 legacy=0x7f4b0c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b10963700 1 -- 172.21.15.26:0/2253282447 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c1625c0 con 0x7f4b0c1478c0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b11164700 1 --2- 172.21.15.26:0/2253282447 >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 0x7f4b0c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b10963700 1 --2- 172.21.15.26:0/2253282447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c1478c0 0x7f4b0c0b57b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c06c040 tx=0x7f4b00009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9f5eb7538cba56 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b0b7fe700 1 -- 172.21.15.26:0/2253282447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00011830 con 0x7f4b0c1478c0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b0b7fe700 1 -- 172.21.15.26:0/2253282447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b00011990 con 0x7f4b0c1478c0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b0b7fe700 1 -- 172.21.15.26:0/2253282447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00011d20 con 0x7f4b0c1478c0 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b12bc7700 1 -- 172.21.15.26:0/2253282447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c1478c0 msgr2=0x7f4b0c0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.259+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/2253282447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c1478c0 0x7f4b0c0b57b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c06c040 tx=0x7f4b00009b20 comp rx=0 tx=0).stop 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 -- 172.21.15.26:0/2253282447 shutdown_connections 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/2253282447 >> v2:172.21.15.93:3300/0 conn(0x7f4b0c06b6e0 0x7f4b0c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/2253282447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c1478c0 0x7f4b0c0b57b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 -- 172.21.15.26:0/2253282447 >> 172.21.15.26:0/2253282447 conn(0x7f4b0c0d0aa0 msgr2=0x7f4b0c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 -- 172.21.15.26:0/2253282447 shutdown_connections 2024-01-16T14:55:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.260+0000 7f4b12bc7700 1 -- 172.21.15.26:0/2253282447 wait complete. 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.261+0000 7f4b12bc7700 1 Processor -- start 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.261+0000 7f4b12bc7700 1 -- start start 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4b0c1478c0 0x7f4b0c1e7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b0c161a90 0x7f4b0c140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1e81f0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1e84d0 con 0x7f4b0c161a90 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b12bc7700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1e87b0 con 0x7f4b0c1478c0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41456/0 (socket says 172.21.15.26:41456) 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 -- 172.21.15.26:0/3877914508 learned_addr learned my addr 172.21.15.26:0/3877914508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b11164700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b0c161a90 0x7f4b0c140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b0bfff700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4b0c1478c0 0x7f4b0c1e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 -- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4b0c1478c0 msgr2=0x7f4b0c1e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4b0c1478c0 0x7f4b0c1e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 -- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b0c161a90 msgr2=0x7f4b0c140d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b0c161a90 0x7f4b0c140d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.262+0000 7f4b10963700 1 -- 172.21.15.26:0/3877914508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000094a0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b10963700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4b000090b0 tx=0x7f4b00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000036a0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0001f070 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c1e8a90 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003a60 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.263+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c141ab0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.264+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c1e8fb0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.264+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4b00003bc0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.265+0000 7f4b097fa700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 0x7f4af006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.265+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b00021070 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.265+0000 7f4b0bfff700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 0x7f4af006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.265+0000 7f4b0bfff700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 0x7f4af006d810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c1f73a0 tx=0x7f4af8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.270+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b000563b0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.573+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4b0c075480 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.575+0000 7f4b097fa700 1 -- 172.21.15.26:0/3877914508 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4b000599d0 con 0x7f4b0c06b6e0 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQCDmKZl8JqJORAAXPhIkgN/hhjJtdjFgPeRLQ== 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-16T14:55:48.558 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 msgr2=0x7f4af006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 0x7f4af006d810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c1f73a0 tx=0x7f4af8009040 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 msgr2=0x7f4b0c15d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4b000090b0 tx=0x7f4b00004750 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 shutdown_connections 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4b0c161a90 0x7f4b0c140d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f4af006b350 0x7f4af006d810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0c06b6e0 0x7f4b0c15d790 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 --2- 172.21.15.26:0/3877914508 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4b0c1478c0 0x7f4b0c1e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 >> 172.21.15.26:0/3877914508 conn(0x7f4b0c0d0aa0 msgr2=0x7f4b0c1451e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 shutdown_connections 2024-01-16T14:55:48.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.578+0000 7f4b12bc7700 1 -- 172.21.15.26:0/3877914508 wait complete. 2024-01-16T14:55:48.562 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-16T14:55:48.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-16T14:55:48.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.728+0000 7f03e6d49700 1 Processor -- start 2024-01-16T14:55:48.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.728+0000 7f03e6d49700 1 -- start start 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03e006b620 0x7f03e00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f03e00b5ab0 0x7f03e00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e01478c0 0x7f03e0161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e0071f50 con 0x7f03e006b620 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e0072230 con 0x7f03e00b5ab0 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e6d49700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0072510 con 0x7f03e01478c0 2024-01-16T14:55:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03e006b620 0x7f03e00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03e006b620 0x7f03e00b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41472/0 (socket says 172.21.15.26:41472) 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 -- 172.21.15.26:0/4210504825 learned_addr learned my addr 172.21.15.26:0/4210504825 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03dffff700 1 --2- 172.21.15.26:0/4210504825 >> v2:172.21.15.93:3300/0 conn(0x7f03e00b5ab0 0x7f03e00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e52e6700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e01478c0 0x7f03e0161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 -- 172.21.15.26:0/4210504825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e01478c0 msgr2=0x7f03e0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e01478c0 0x7f03e0161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 -- 172.21.15.26:0/4210504825 >> v2:172.21.15.93:3300/0 conn(0x7f03e00b5ab0 msgr2=0x7f03e00b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/4210504825 >> v2:172.21.15.93:3300/0 conn(0x7f03e00b5ab0 0x7f03e00b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.729+0000 7f03e4ae5700 1 -- 172.21.15.26:0/4210504825 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00727f0 con 0x7f03e006b620 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e006b620 0x7f03e00b3a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03e006d6f0 tx=0x7f03d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e906b26dc9415d50 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03df7fe700 1 -- 172.21.15.26:0/4210504825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4011830 con 0x7f03e006b620 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03df7fe700 1 -- 172.21.15.26:0/4210504825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d4011990 con 0x7f03e006b620 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03df7fe700 1 -- 172.21.15.26:0/4210504825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4011d20 con 0x7f03e006b620 2024-01-16T14:55:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e6d49700 1 -- 172.21.15.26:0/4210504825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e006b620 msgr2=0x7f03e00b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e6d49700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e006b620 0x7f03e00b3a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03e006d6f0 tx=0x7f03d4009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e6d49700 1 -- 172.21.15.26:0/4210504825 shutdown_connections 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e6d49700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e01478c0 0x7f03e0161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.730+0000 7f03e6d49700 1 --2- 172.21.15.26:0/4210504825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e006b620 0x7f03e00b3a40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.731+0000 7f03e6d49700 1 --2- 172.21.15.26:0/4210504825 >> v2:172.21.15.93:3300/0 conn(0x7f03e00b5ab0 0x7f03e00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.731+0000 7f03e6d49700 1 -- 172.21.15.26:0/4210504825 >> 172.21.15.26:0/4210504825 conn(0x7f03e00b2740 msgr2=0x7f03e00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.731+0000 7f03e6d49700 1 -- 172.21.15.26:0/4210504825 shutdown_connections 2024-01-16T14:55:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.731+0000 7f03e6d49700 1 -- 172.21.15.26:0/4210504825 wait complete. 2024-01-16T14:55:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.732+0000 7f03e6d49700 1 Processor -- start 2024-01-16T14:55:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.732+0000 7f03e6d49700 1 -- start start 2024-01-16T14:55:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03e006b620 0x7f03e01e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e00b5ab0 0x7f03e01365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01e2990 con 0x7f03e01478c0 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e01dbdc0 con 0x7f03e006b620 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e6d49700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e0136b20 con 0x7f03e00b5ab0 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41484/0 (socket says 172.21.15.26:41484) 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 -- 172.21.15.26:0/1735557294 learned_addr learned my addr 172.21.15.26:0/1735557294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03dffff700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e00b5ab0 0x7f03e01365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03e006b620 0x7f03e01e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 -- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e00b5ab0 msgr2=0x7f03e01365e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e00b5ab0 0x7f03e01365e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 -- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03e006b620 msgr2=0x7f03e01e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03e006b620 0x7f03e01e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.733+0000 7f03e52e6700 1 -- 172.21.15.26:0/1735557294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009550 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03e52e6700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f03d800efc0 tx=0x7f03d800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800cad0 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d800f040 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d800ce70 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e01e1d10 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.734+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e01e22d0 con 0x7f03e01478c0 2024-01-16T14:55:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.735+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e0075480 con 0x7f03e01478c0 2024-01-16T14:55:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.735+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f03d8014480 con 0x7f03e01478c0 2024-01-16T14:55:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.736+0000 7f03dd7fa700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 0x7f03c406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.736+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03d808a350 con 0x7f03e01478c0 2024-01-16T14:55:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.736+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 0x7f03c406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:48.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.737+0000 7f03e4ae5700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 0x7f03c406d810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009c10 tx=0x7f03d4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:48.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.741+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03d8055270 con 0x7f03e01478c0 2024-01-16T14:55:48.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.997+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03e006bd40 con 0x7f03e01478c0 2024-01-16T14:55:48.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:48.998+0000 7f03dd7fa700 1 -- 172.21.15.26:0/1735557294 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f03d8058890 con 0x7f03e01478c0 2024-01-16T14:55:48.981 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:55:48.981 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/923856271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923856271},{"type":"v1","addr":"172.21.15.26:6834","nonce":923856271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3873344155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3873344155},{"type":"v1","addr":"172.21.15.26:6835","nonce":3873344155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4969,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6836/396498543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":396498543},{"type":"v1","addr":"172.21.15.93:6836","nonce":396498543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-16T14:55:42.618646+0000","modified":"2024-01-16T14:55:45.863082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4972":{"gid":4972,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.93:6837/406936670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6835","nonce":406936670},{"type":"v1","addr":"172.21.15.93:6837","nonce":406936670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-16T14:55:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 msgr2=0x7f03c406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 0x7f03c406d810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009c10 tx=0x7f03d4008030 comp rx=0 tx=0).stop 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 msgr2=0x7f03e01f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f03d800efc0 tx=0x7f03d800c0c0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 shutdown_connections 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f03c406b350 0x7f03c406d810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03e01478c0 0x7f03e01f36d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03e006b620 0x7f03e01e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 --2- 172.21.15.26:0/1735557294 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03e00b5ab0 0x7f03e01365e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 >> 172.21.15.26:0/1735557294 conn(0x7f03e00b2740 msgr2=0x7f03e00b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 shutdown_connections 2024-01-16T14:55:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.000+0000 7f03e6d49700 1 -- 172.21.15.26:0/1735557294 wait complete. 2024-01-16T14:55:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2024-01-16T14:55:49.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-16T14:55:42.618646+0000', 'modified': '2024-01-16T14:55:45.863082+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4972': {'gid': 4972, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.93:6837/406936670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6835', 'nonce': 406936670}, {'type': 'v1', 'addr': '172.21.15.93:6837', 'nonce': 406936670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-16T14:55:49.400 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-16T14:55:49.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-16T14:55:49.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.565+0000 7faaec4ee700 1 Processor -- start 2024-01-16T14:55:49.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.565+0000 7faaec4ee700 1 -- start start 2024-01-16T14:55:49.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faaec4ee700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faae40d2b60 0x7faae415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faaec4ee700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40d4ce0 con 0x7faae40d0510 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faaec4ee700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae415f810 con 0x7faae415e3b0 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faaec4ee700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40d1630 con 0x7faae40d2b60 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faaeaa8b700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faae40d2b60 0x7faae415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faae9a89700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faae40d0510 0x7faae40ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43962/0 (socket says 172.21.15.26:43962) 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faae9a89700 1 -- 172.21.15.26:0/1951404507 learned_addr learned my addr 172.21.15.26:0/1951404507 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.566+0000 7faae9288700 1 -- 172.21.15.26:0/1951404507 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247533161 0 0) 0x7faae40d4ce0 con 0x7faae40d0510 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faae9288700 1 -- 172.21.15.26:0/1951404507 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad00035b0 con 0x7faae40d0510 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faaeaa8b700 1 -- 172.21.15.26:0/1951404507 >> v1:172.21.15.93:6789/0 conn(0x7faae415e3b0 legacy=0x7faae40d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faaeaa8b700 1 -- 172.21.15.26:0/1951404507 >> v1:172.21.15.26:6789/0 conn(0x7faae40d0510 legacy=0x7faae40ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faaeaa8b700 1 -- 172.21.15.26:0/1951404507 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40d09b0 con 0x7faae40d2b60 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/1951404507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae415f2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35c72c687787c3b server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faae9288700 1 -- 172.21.15.26:0/1951404507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00e070 con 0x7faae40d2b60 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faae9288700 1 -- 172.21.15.26:0/1951404507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc00bb20 con 0x7faae40d2b60 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.567+0000 7faae9288700 1 -- 172.21.15.26:0/1951404507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc012430 con 0x7faae40d2b60 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 -- 172.21.15.26:0/1951404507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 msgr2=0x7faae415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 --2- 172.21.15.26:0/1951404507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae415f2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7faadc009d90 tx=0x7faadc00b330 comp rx=0 tx=0).stop 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 -- 172.21.15.26:0/1951404507 shutdown_connections 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 --2- 172.21.15.26:0/1951404507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae415f2d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 -- 172.21.15.26:0/1951404507 >> 172.21.15.26:0/1951404507 conn(0x7faae40d3e70 msgr2=0x7faae40d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 -- 172.21.15.26:0/1951404507 shutdown_connections 2024-01-16T14:55:49.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.568+0000 7faaec4ee700 1 -- 172.21.15.26:0/1951404507 wait complete. 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.569+0000 7faaec4ee700 1 Processor -- start 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.569+0000 7faaec4ee700 1 -- start start 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae40d0510 0x7faae4152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae415e3b0 0x7faae40b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4159e20 con 0x7faae40d2b60 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b7770 con 0x7faae415e3b0 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaec4ee700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae4152f50 con 0x7faae40d0510 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41506/0 (socket says 172.21.15.26:41506) 2024-01-16T14:55:49.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 -- 172.21.15.26:0/2300905484 learned_addr learned my addr 172.21.15.26:0/2300905484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaea28a700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae40d0510 0x7faae4152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae415e3b0 0x7faae40b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 -- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae40d0510 msgr2=0x7faae4152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae40d0510 0x7faae4152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 -- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae415e3b0 msgr2=0x7faae40b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae415e3b0 0x7faae40b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.570+0000 7faae9a89700 1 -- 172.21.15.26:0/2300905484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc009a70 con 0x7faae40d2b60 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faae9a89700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faad800c700 tx=0x7faad800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faad6ffd700 1 -- 172.21.15.26:0/2300905484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8005d70 con 0x7faae40d2b60 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faad6ffd700 1 -- 172.21.15.26:0/2300905484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad80070d0 con 0x7faae40d2b60 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae40b7c00 con 0x7faae40d2b60 2024-01-16T14:55:49.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faad6ffd700 1 -- 172.21.15.26:0/2300905484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8007460 con 0x7faae40d2b60 2024-01-16T14:55:49.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.571+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae40b6e40 con 0x7faae40d2b60 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.573+0000 7faad6ffd700 1 -- 172.21.15.26:0/2300905484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faad80075c0 con 0x7faae40d2b60 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.573+0000 7faad6ffd700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 0x7faac806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.573+0000 7faad6ffd700 1 -- 172.21.15.26:0/2300905484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faad8088f40 con 0x7faae40d2b60 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.574+0000 7faaea28a700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 0x7faac806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.574+0000 7faaea28a700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 0x7faac806d810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faae0007fd0 tx=0x7faae0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.575+0000 7faaf86eab80 1 Processor -- start 2024-01-16T14:55:49.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.575+0000 7faaf86eab80 1 -- start start 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0b5dd0 0x558b0f0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f02b480 con 0x558b0f0e0410 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f011770 con 0x558b0f0b5dd0 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0f00f090 con 0x558b0f043ce0 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41508/0 (socket says 172.21.15.26:41508) 2024-01-16T14:55:49.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 -- 172.21.15.26:0/882025465 learned_addr learned my addr 172.21.15.26:0/882025465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaea28a700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0b5dd0 0x558b0f0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 -- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 msgr2=0x558b0f0fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0fecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 -- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0b5dd0 msgr2=0x558b0f0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0b5dd0 0x558b0f0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 -- 172.21.15.26:0/882025465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b0ee33b10 con 0x558b0f0e0410 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faaea28a700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.576+0000 7faae9a89700 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faad8087520 tx=0x7faad800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e351d4d495a77b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.578+0000 7faac7fff700 1 -- 172.21.15.26:0/882025465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8058200 con 0x558b0f0e0410 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.578+0000 7faac7fff700 1 -- 172.21.15.26:0/882025465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8057b00 con 0x558b0f0e0410 2024-01-16T14:55:49.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.578+0000 7faac7fff700 1 -- 172.21.15.26:0/882025465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad805bc50 con 0x558b0f0e0410 2024-01-16T14:55:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 -- 172.21.15.26:0/882025465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 msgr2=0x558b0f0db190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faad8087520 tx=0x7faad800cf90 comp rx=0 tx=0).stop 2024-01-16T14:55:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 -- 172.21.15.26:0/882025465 shutdown_connections 2024-01-16T14:55:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0b5dd0 0x558b0f0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 --2- 172.21.15.26:0/882025465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0db190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 -- 172.21.15.26:0/882025465 >> 172.21.15.26:0/882025465 conn(0x558b0efe5410 msgr2=0x558b0f0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 -- 172.21.15.26:0/882025465 shutdown_connections 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.579+0000 7faaf86eab80 1 -- 172.21.15.26:0/882025465 wait complete. 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.580+0000 7faaf86eab80 1 -- start start 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.580+0000 7faaf86eab80 1 Processor -- start 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.581+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.581+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.581+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f101250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.581+0000 7faaf86eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f00a980 con 0x558b0f0b5dd0 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.581+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0efaa000 con 0x558b0f0e0410 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0efa9ca0 con 0x558b0f043ce0 2024-01-16T14:55:49.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faaf86eab80 10 client.0 _ll_register_callbacks cb 0x558b0ee0d350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41510/0 (socket says 172.21.15.26:41510) 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 -- 172.21.15.26:0/536225548 learned_addr learned my addr 172.21.15.26:0/536225548 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faaea28a700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 -- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 msgr2=0x558b0f0e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 -- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 msgr2=0x558b0f101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 -- 172.21.15.26:0/536225548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0001cc0 con 0x558b0f0b5dd0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faae9a89700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faad8053fb0 tx=0x7faad8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477167ad2ed6cee3 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faac57fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8058200 con 0x558b0f0b5dd0 2024-01-16T14:55:49.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faad805fe00 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.582+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b0f02a450 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad805bc50 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 10 client.5220 fetch_fsmap learned FSMap version 17 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x7faae40b6e40 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558b0f02a450 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faad8076300 con 0x558b0f0b5dd0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 10 client.5220 fetch_fsmap finished waiting for FSMap version 17 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 10 client.5220 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 20 client.5220 populate_metadata read hostname 'smithi026' 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0ff1a0 0x558b0f0ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 4 client.5220 mds_command: new command op to 4972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 --> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b0f003960 con 0x558b0f0ff1a0 2024-01-16T14:55:49.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.583+0000 7faaea28a700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0ff1a0 0x558b0f0ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.584+0000 7faac57fa700 10 client.5220 ms_handle_connect on v2:172.21.15.93:6835/406936670 2024-01-16T14:55:49.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.584+0000 7faaea28a700 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0ff1a0 0x558b0f0ff580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a226db7ac18dd46d server_cookie=f53188cf39578820 in_seq=0 out_seq=0 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.584+0000 7faac57fa700 1 -- 172.21.15.26:0/536225548 <== mds.0 v2:172.21.15.93:6835/406936670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b0f003960 con 0x558b0f0ff1a0 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.584+0000 7faac57fa700 10 client.5220 handle_command_reply: tid=0 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.584+0000 7faaf86eab80 1 client.5220 shutdown 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 msgr2=0x558b0f100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faad8053fb0 tx=0x7faad8000ed0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 shutdown_connections 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f043ce0 0x558b0f0e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0ff1a0 0x558b0f0ff580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 --2- 172.21.15.26:0/536225548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0b5dd0 0x558b0f100d10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 >> 172.21.15.26:0/536225548 conn(0x558b0efe5410 msgr2=0x558b0f0da470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faac57fa700 0 client.5220 ms_handle_reset on v2:172.21.15.93:6835/406936670 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 shutdown_connections 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 1 -- 172.21.15.26:0/536225548 wait complete. 2024-01-16T14:55:49.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.585+0000 7faaf86eab80 20 client.5220 trim_cache size 0 max 16384 2024-01-16T14:55:49.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.604+0000 7faaf86eab80 1 Processor -- start 2024-01-16T14:55:49.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.604+0000 7faaf86eab80 1 -- start start 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f0e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0ee89650 con 0x558b0f043ce0 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f0e4430 con 0x558b0f0e0410 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0f0b4450 con 0x558b0f0b5dd0 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41516/0 (socket says 172.21.15.26:41516) 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 -- 172.21.15.26:0/451278526 learned_addr learned my addr 172.21.15.26:0/451278526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faae9a89700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f0e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 -- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 msgr2=0x558b0f0d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 -- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 msgr2=0x558b0f0e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f0e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaea28a700 1 -- 172.21.15.26:0/451278526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b0efcc290 con 0x558b0f043ce0 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.605+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f0e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:55:49.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaea28a700 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faae0000f60 tx=0x7faae0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bd62cebe4fe9be server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faad4ff9700 1 -- 172.21.15.26:0/451278526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0028070 con 0x558b0f043ce0 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faad4ff9700 1 -- 172.21.15.26:0/451278526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae000fbd0 con 0x558b0f043ce0 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faad4ff9700 1 -- 172.21.15.26:0/451278526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000fd40 con 0x558b0f043ce0 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 -- 172.21.15.26:0/451278526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 msgr2=0x558b0f100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faae0000f60 tx=0x7faae0012000 comp rx=0 tx=0).stop 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 -- 172.21.15.26:0/451278526 shutdown_connections 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f0e0410 0x558b0f0e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f043ce0 0x558b0f100b50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 --2- 172.21.15.26:0/451278526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 -- 172.21.15.26:0/451278526 >> 172.21.15.26:0/451278526 conn(0x558b0efe5410 msgr2=0x558b0f0b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 -- 172.21.15.26:0/451278526 shutdown_connections 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.606+0000 7faaf86eab80 1 -- 172.21.15.26:0/451278526 wait complete. 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.607+0000 7faaf86eab80 1 -- start start 2024-01-16T14:55:49.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.607+0000 7faaf86eab80 1 Processor -- start 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f043ce0 0x558b0f15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f15cf00 con 0x558b0f0e0410 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0f0f3e70 con 0x558b0f043ce0 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0f0e0000 con 0x558b0f0b5dd0 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaf86eab80 10 client.0 _ll_register_callbacks cb 0x558b0ee0d350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-16T14:55:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41530/0 (socket says 172.21.15.26:41530) 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 -- 172.21.15.26:0/865187603 learned_addr learned my addr 172.21.15.26:0/865187603 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaea28a700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f043ce0 0x558b0f15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faae9a89700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 -- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 msgr2=0x558b0f0b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 -- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f043ce0 msgr2=0x558b0f15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f043ce0 0x558b0f15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 -- 172.21.15.26:0/865187603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae000f800 con 0x558b0f0e0410 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faaeaa8b700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faae40d18f0 tx=0x7faadc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a60c35dc4084fc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faac57fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.609+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc012de0 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faadc004040 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faad8057b00 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7faadc004200 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 10 client.5226 fetch_fsmap learned FSMap version 17 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558b0ee8bc80 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8057b00 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faadc026070 con 0x558b0f0e0410 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 10 client.5226 fetch_fsmap finished waiting for FSMap version 17 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 10 client.5226 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 20 client.5226 populate_metadata read hostname 'smithi026' 2024-01-16T14:55:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0e29a0 0x558b0f0e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 4 client.5226 mds_command: new command op to 4972 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 --> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558b0f003680 con 0x558b0f0e29a0 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.610+0000 7faaea28a700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0e29a0 0x558b0f0e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faac57fa700 10 client.5226 ms_handle_connect on v2:172.21.15.93:6835/406936670 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaea28a700 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0e29a0 0x558b0f0e2d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d7f23947633e68 server_cookie=88741715d9610439 in_seq=0 out_seq=0 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faac57fa700 1 -- 172.21.15.26:0/865187603 <== mds.0 v2:172.21.15.93:6835/406936670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558b0f003680 con 0x558b0f0e29a0 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faac57fa700 10 client.5226 handle_command_reply: tid=0 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 client.5226 shutdown 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 msgr2=0x558b0f0f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faae40d18f0 tx=0x7faadc004b40 comp rx=0 tx=0).stop 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 shutdown_connections 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:6835/406936670,v1:172.21.15.93:6837/406936670] conn(0x558b0f0e29a0 0x558b0f0e2d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558b0f0e0410 0x558b0f0f31c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b0f043ce0 0x558b0f15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faac57fa700 0 client.5226 ms_handle_reset on v2:172.21.15.93:6835/406936670 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 --2- 172.21.15.26:0/865187603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558b0f0b5dd0 0x558b0f0b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 >> 172.21.15.26:0/865187603 conn(0x558b0efe5410 msgr2=0x558b0f0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 shutdown_connections 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 1 -- 172.21.15.26:0/865187603 wait complete. 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.611+0000 7faaf86eab80 20 client.5226 trim_cache size 0 max 16384 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:55:49.601 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "d39fe2d6-b323-4e7f-9f15-004380a58c0d", 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4972, 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.1847313650000002, 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 17, 2024-01-16T14:55:49.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 32, 2024-01-16T14:55:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 32, 2024-01-16T14:55:49.603 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 14.311428045 2024-01-16T14:55:49.603 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 msgr2=0x7faac806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 0x7faac806d810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faae0007fd0 tx=0x7faae0009040 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 msgr2=0x7faae40b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faad800c700 tx=0x7faad800ca10 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 shutdown_connections 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae415e3b0 0x7faae40b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae40d0510 0x7faae4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faac806b350 0x7faac806d810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 --2- 172.21.15.26:0/2300905484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40d2b60 0x7faae40b82b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 >> 172.21.15.26:0/2300905484 conn(0x7faae40d3e70 msgr2=0x7faae406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 shutdown_connections 2024-01-16T14:55:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.612+0000 7faaec4ee700 1 -- 172.21.15.26:0/2300905484 wait complete. 2024-01-16T14:55:49.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.761+0000 7fb47280d700 1 Processor -- start 2024-01-16T14:55:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.761+0000 7fb47280d700 1 -- start start 2024-01-16T14:55:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c15be40 0x7fb46c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb46c0d0ec0 0x7fb46c0b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb46c0b1a90 0x7fb46c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0b1130 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c067420 con 0x7fb46c0b1a90 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.762+0000 7fb47280d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0af940 con 0x7fb46c15be40 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb46c0d0ec0 0x7fb46c0b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb46c0d0ec0 0x7fb46c0b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41532/0 (socket says 172.21.15.26:41532) 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 -- 172.21.15.26:0/3482181298 learned_addr learned my addr 172.21.15.26:0/3482181298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46bfff700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c15be40 0x7fb46c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb470daa700 1 --2- 172.21.15.26:0/3482181298 >> v2:172.21.15.93:3300/0 conn(0x7fb46c0b1a90 0x7fb46c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 -- 172.21.15.26:0/3482181298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c15be40 msgr2=0x7fb46c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c15be40 0x7fb46c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 -- 172.21.15.26:0/3482181298 >> v2:172.21.15.93:3300/0 conn(0x7fb46c0b1a90 msgr2=0x7fb46c06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/3482181298 >> v2:172.21.15.93:3300/0 conn(0x7fb46c0b1a90 0x7fb46c06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 -- 172.21.15.26:0/3482181298 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0b0ab0 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c0b1550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098f0 tx=0x7fb454009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542f7ae9a1a060f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46affd700 1 -- 172.21.15.26:0/3482181298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454011830 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46affd700 1 -- 172.21.15.26:0/3482181298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454011990 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.763+0000 7fb46affd700 1 -- 172.21.15.26:0/3482181298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454011d20 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 -- 172.21.15.26:0/3482181298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 msgr2=0x7fb46c0b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c0b1550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb4540098f0 tx=0x7fb454009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 -- 172.21.15.26:0/3482181298 shutdown_connections 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 --2- 172.21.15.26:0/3482181298 >> v2:172.21.15.93:3300/0 conn(0x7fb46c0b1a90 0x7fb46c06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c15be40 0x7fb46c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 --2- 172.21.15.26:0/3482181298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c0b1550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 -- 172.21.15.26:0/3482181298 >> 172.21.15.26:0/3482181298 conn(0x7fb46c0717f0 msgr2=0x7fb46c0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 -- 172.21.15.26:0/3482181298 shutdown_connections 2024-01-16T14:55:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.764+0000 7fb47280d700 1 -- 172.21.15.26:0/3482181298 wait complete. 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.765+0000 7fb47280d700 1 Processor -- start 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.766+0000 7fb47280d700 1 -- start start 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.766+0000 7fb47280d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb46c15be40 0x7fb46c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c159b90 0x7fb46c0ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c15a020 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0c9f20 con 0x7fb46c15be40 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c155960 con 0x7fb46c159b90 2024-01-16T14:55:49.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41544/0 (socket says 172.21.15.26:41544) 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 -- 172.21.15.26:0/1180139216 learned_addr learned my addr 172.21.15.26:0/1180139216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb46c15be40 0x7fb46c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb470daa700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c159b90 0x7fb46c0ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 -- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c159b90 msgr2=0x7fb46c0ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c159b90 0x7fb46c0ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 -- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb46c15be40 msgr2=0x7fb46c159650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb46c15be40 0x7fb46c159650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 -- 172.21.15.26:0/1180139216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454009550 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb46bfff700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e8a0 tx=0x7fb45c00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c009590 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c013680 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c0c9630 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.767+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0139f0 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.768+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0c9bb0 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.768+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c0aedf0 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.769+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb45c013b50 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.769+0000 7fb468ff9700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 0x7fb44c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.770+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb45c089b00 con 0x7fb46c0d0ec0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.770+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 0x7fb44c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:49.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.770+0000 7fb46b7fe700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 0x7fb44c06d810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0d1300 tx=0x7fb454008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:49.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:49.774+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb45c08be60 con 0x7fb46c0d0ec0 2024-01-16T14:55:50.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.036+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb46c075480 con 0x7fb46c0d0ec0 2024-01-16T14:55:50.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.036+0000 7fb468ff9700 1 -- 172.21.15.26:0/1180139216 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb45c054850 con 0x7fb46c0d0ec0 2024-01-16T14:55:50.020 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 msgr2=0x7fb44c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 0x7fb44c06d810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0d1300 tx=0x7fb454008030 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 msgr2=0x7fb46c1f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00e8a0 tx=0x7fb45c00ebb0 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 shutdown_connections 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46c159b90 0x7fb46c0ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb44c06b350 0x7fb44c06d810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb46c0d0ec0 0x7fb46c1f2f80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 --2- 172.21.15.26:0/1180139216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb46c15be40 0x7fb46c159650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 >> 172.21.15.26:0/1180139216 conn(0x7fb46c0717f0 msgr2=0x7fb46c0cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 shutdown_connections 2024-01-16T14:55:50.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:50.039+0000 7fb47280d700 1 -- 172.21.15.26:0/1180139216 wait complete. 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-16T14:55:50.035 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-16T14:55:50.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:50.036 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T14:55:50.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.036 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-16T14:55:50.055 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:50.056 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42375sec preferred_lft 42375sec 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 9a:53:22:ce:db:98 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.057 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::b438:a1ff:fe82:ed32/64 scope link 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:9e:c4:45:96:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c49e:c4ff:fe45:96b6/64 scope link 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 9a:53:22:ce:db:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::9853:22ff:fece:db98/64 scope link 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:50.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.058 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:50.127 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-16T14:55:50.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.139 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:50.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-16T14:55:50.205 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-16T14:55:50.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.330 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-16T14:55:50.350 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-16T14:55:50.350 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-16T14:55:50.350 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-16T14:55:50.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-16T14:55:50.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:50.425 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:50.487 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-16T14:55:50.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:50.487 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:50.548 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:50.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:50.647 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:50.647 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:50.647 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:50.658 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:50.658 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:50.658 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:50.658 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:50.658 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:50.678 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:50.681 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:50.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-16T14:55:50.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:50.714 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:50.815 INFO:teuthology.orchestra.run.smithi026.stdout:client5158 2024-01-16T14:55:50.815 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:50.815 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:50.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-16T14:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.040+0000 7f9187c10700 1 Processor -- start 2024-01-16T14:55:51.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.040+0000 7f9187c10700 1 -- start start 2024-01-16T14:55:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f9187c10700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91800d0250 0x7f91800d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f9187c10700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f9187c10700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91800c5bd0 con 0x7f91800cc780 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f9187c10700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91800cd310 con 0x7f91800d0250 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f9187c10700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91800d0b70 con 0x7f91800c6060 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f91859ac700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91800d0250 0x7f91800d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f91859ac700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91800d0250 0x7f91800d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41556/0 (socket says 172.21.15.26:41556) 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.041+0000 7f91859ac700 1 -- 172.21.15.26:0/2695849838 learned_addr learned my addr 172.21.15.26:0/2695849838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91861ad700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91859ac700 1 -- 172.21.15.26:0/2695849838 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 msgr2=0x7f91800c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91859ac700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91859ac700 1 -- 172.21.15.26:0/2695849838 >> v1:172.21.15.93:6789/0 conn(0x7f91800cc780 legacy=0x7f91800ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91859ac700 1 -- 172.21.15.26:0/2695849838 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800cdb30 con 0x7f91800d0250 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91859ac700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f91800d0630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3db472512e27dd server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91849aa700 1 -- 172.21.15.26:0/2695849838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00b8a0 con 0x7f91800d0250 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91849aa700 1 -- 172.21.15.26:0/2695849838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c00ba00 con 0x7f91800d0250 2024-01-16T14:55:51.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.042+0000 7f91849aa700 1 -- 172.21.15.26:0/2695849838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00bd90 con 0x7f91800d0250 2024-01-16T14:55:51.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 -- 172.21.15.26:0/2695849838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 msgr2=0x7f91800d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f91800d0630 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f917c0098f0 tx=0x7f917c009c00 comp rx=0 tx=0).stop 2024-01-16T14:55:51.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 -- 172.21.15.26:0/2695849838 shutdown_connections 2024-01-16T14:55:51.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 --2- 172.21.15.26:0/2695849838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f91800d0630 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 -- 172.21.15.26:0/2695849838 >> 172.21.15.26:0/2695849838 conn(0x7f91800b3800 msgr2=0x7f91800b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 -- 172.21.15.26:0/2695849838 shutdown_connections 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.043+0000 7f9187c10700 1 -- 172.21.15.26:0/2695849838 wait complete. 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.044+0000 7f9187c10700 1 Processor -- start 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.044+0000 7f9187c10700 1 -- start start 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91800cc780 0x7f91800ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91801f1590 con 0x7f91800d0250 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180147680 con 0x7f91800cc780 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f9187c10700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9180147960 con 0x7f91800c6060 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41562/0 (socket says 172.21.15.26:41562) 2024-01-16T14:55:51.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 -- 172.21.15.26:0/4123614178 learned_addr learned my addr 172.21.15.26:0/4123614178 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91859ac700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91851ab700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91800cc780 0x7f91800ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 -- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 msgr2=0x7f91800b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 -- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91800cc780 msgr2=0x7f91800ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91800cc780 0x7f91800ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.045+0000 7f91861ad700 1 -- 172.21.15.26:0/4123614178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009550 con 0x7f91800d0250 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f91861ad700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9178007e60 tx=0x7f917800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800ed30 con 0x7f91800d0250 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9178014420 con 0x7f91800d0250 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9180147c40 con 0x7f91800d0250 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780147d0 con 0x7f91800d0250 2024-01-16T14:55:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.046+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9180148200 con 0x7f91800d0250 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.047+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91800d14b0 con 0x7f91800d0250 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.047+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9178014930 con 0x7f91800d0250 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.048+0000 7f91767fc700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 0x7f916406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.048+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f917808a140 con 0x7f91800d0250 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.048+0000 7f91859ac700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 0x7f916406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.049+0000 7f91859ac700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 0x7f916406d810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f917c009c40 tx=0x7f917c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:51.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.053+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9178055060 con 0x7f91800d0250 2024-01-16T14:55:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.316+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9180148420 con 0x7f91800d0250 2024-01-16T14:55:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.316+0000 7f91767fc700 1 -- 172.21.15.26:0/4123614178 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9178058680 con 0x7f91800d0250 2024-01-16T14:55:51.299 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-16T14:55:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 msgr2=0x7f916406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 0x7f916406d810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f917c009c40 tx=0x7f917c008030 comp rx=0 tx=0).stop 2024-01-16T14:55:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 msgr2=0x7f9180148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9178007e60 tx=0x7f917800e3c0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 shutdown_connections 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f916406b350 0x7f916406d810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91800d0250 0x7f9180148720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f91800c6060 0x7f91800b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 --2- 172.21.15.26:0/4123614178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91800cc780 0x7f91800ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 >> 172.21.15.26:0/4123614178 conn(0x7f91800b3800 msgr2=0x7f91800cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 shutdown_connections 2024-01-16T14:55:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:51.319+0000 7f9187c10700 1 -- 172.21.15.26:0/4123614178 wait complete. 2024-01-16T14:55:51.315 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-16T14:55:51.315 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-16T14:55:51.315 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2024-01-16T14:55:51.315 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-16T14:55:51.315 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:51.316 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-16T14:55:51.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-01-16T14:55:51.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-16T14:55:51.316 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T14:55:51.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.316 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2024-01-16T14:55:51.338 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-16T14:55:51.338 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-16T14:55:51.338 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2024-01-16T14:55:51.338 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.338 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42407sec preferred_lft 42407sec 2024-01-16T14:55:51.339 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::fd4b:da16:302c:bf58/64 scope link noprefixroute 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 3a:b8:45:9d:25:3b brd ff:ff:ff:ff:ff:ff 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::2432:a9ff:fe27:8dbe/64 scope link 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether da:53:f5:87:db:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::d853:f5ff:fe87:dbed/64 scope link 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 3a:b8:45:9d:25:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-16T14:55:51.340 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::38b8:45ff:fe9d:253b/64 scope link 2024-01-16T14:55:51.341 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2024-01-16T14:55:51.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.341 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-16T14:55:51.410 INFO:teuthology.orchestra.run.smithi093.stdout:1 2024-01-16T14:55:51.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.412 DEBUG:teuthology.orchestra.run.smithi093:> route 2024-01-16T14:55:51.475 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2024-01-16T14:55:51.475 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-16T14:55:51.475 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-16T14:55:51.475 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-16T14:55:51.476 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-16T14:55:51.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-16T14:55:51.476 DEBUG:teuthology.orchestra.run.smithi093:> ') 2024-01-16T14:55:51.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.590 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2024-01-16T14:55:51.607 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-16T14:55:51.607 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-16T14:55:51.607 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-16T14:55:51.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.607 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2024-01-16T14:55:51.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:51.675 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:51.732 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-16T14:55:51.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-16T14:55:51.733 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:51.789 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-01-16T14:55:51.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.789 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:51.875 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:51.875 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:51.875 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:51.884 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:51.884 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:51.884 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:51.884 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:51.885 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:invalid new device string format 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.93:3300,172.21.15.93:3301" 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: switching to using v1 address with old syntax 2024-01-16T14:55:51.902 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790" 2024-01-16T14:55:51.903 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:6789,172.21.15.93:6789,172.21.15.93:6790:/ 2024-01-16T14:55:51.903 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=d39fe2d6-b323-4e7f-9f15-004380a58c0d" will pass to kernel 2024-01-16T14:55:51.904 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-16T14:55:51.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.905 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-16T14:55:51.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:51.931 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-16T14:55:52.005 INFO:teuthology.orchestra.run.smithi093.stdout:client4871 2024-01-16T14:55:52.005 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2024-01-16T14:55:52.006 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:52.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-16T14:55:52.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.231+0000 7fa0f7f05700 1 Processor -- start 2024-01-16T14:55:52.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.232+0000 7fa0f7f05700 1 -- start start 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.232+0000 7fa0f7f05700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa0f015de60 0x7fa0f006de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f7f05700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f0162af0 con 0x7fa0f015e240 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f7f05700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f0162810 con 0x7fa0f006c6f0 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f7f05700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f006da70 con 0x7fa0f015de60 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f54a0700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0f015e240 0x7fa0f006bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43966/0 (socket says 172.21.15.26:43966) 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f54a0700 1 -- 172.21.15.26:0/1053800261 learned_addr learned my addr 172.21.15.26:0/1053800261 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f5ca1700 1 --2- 172.21.15.26:0/1053800261 >> v2:172.21.15.93:3300/0 conn(0x7fa0f015de60 0x7fa0f006de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:52.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105194122 0 0) 0x7fa0f0162af0 con 0x7fa0f015e240 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0dc0035b0 con 0x7fa0f015e240 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f5ca1700 1 -- 172.21.15.26:0/1053800261 >> v1:172.21.15.93:6789/0 conn(0x7fa0f006c6f0 legacy=0x7fa0f0161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f5ca1700 1 -- 172.21.15.26:0/1053800261 >> v1:172.21.15.26:6789/0 conn(0x7fa0f015e240 legacy=0x7fa0f006bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709210758 0 0) 0x7fa0f0162810 con 0x7fa0f006c6f0 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f5ca1700 1 -- 172.21.15.26:0/1053800261 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f006d3b0 con 0x7fa0f015de60 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.233+0000 7fa0f5ca1700 1 --2- 172.21.15.26:0/1053800261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f015de60 0x7fa0f006de90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009840 tx=0x7fa0e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef431ad0804a3a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4011830 con 0x7fa0f015de60 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4011990 con 0x7fa0f015de60 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f4c9f700 1 -- 172.21.15.26:0/1053800261 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4011d20 con 0x7fa0f015de60 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 -- 172.21.15.26:0/1053800261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f015de60 msgr2=0x7fa0f006de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/1053800261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f015de60 0x7fa0f006de90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009840 tx=0x7fa0e4009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 -- 172.21.15.26:0/1053800261 shutdown_connections 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/1053800261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f015de60 0x7fa0f006de90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 -- 172.21.15.26:0/1053800261 >> 172.21.15.26:0/1053800261 conn(0x7fa0f00b2830 msgr2=0x7fa0f0071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:52.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 -- 172.21.15.26:0/1053800261 shutdown_connections 2024-01-16T14:55:52.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.234+0000 7fa0f7f05700 1 -- 172.21.15.26:0/1053800261 wait complete. 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 Processor -- start 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 -- start start 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f006c6f0 0x7fa0f0152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015e240 0x7fa0f0158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0153610 con 0x7fa0f015de60 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01538f0 con 0x7fa0f006c6f0 2024-01-16T14:55:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.236+0000 7fa0f7f05700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0153bd0 con 0x7fa0f015e240 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41578/0 (socket says 172.21.15.26:41578) 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 -- 172.21.15.26:0/4121549241 learned_addr learned my addr 172.21.15.26:0/4121549241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f5ca1700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f006c6f0 0x7fa0f0152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f64a2700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015e240 0x7fa0f0158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 -- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015e240 msgr2=0x7fa0f0158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015e240 0x7fa0f0158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 -- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f006c6f0 msgr2=0x7fa0f0152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f006c6f0 0x7fa0f0152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 -- 172.21.15.26:0/4121549241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40094a0 con 0x7fa0f015de60 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f54a0700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c400 tx=0x7fa0ec00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec005d70 con 0x7fa0f015de60 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0070d0 con 0x7fa0f015de60 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f0153eb0 con 0x7fa0f015de60 2024-01-16T14:55:52.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec007480 con 0x7fa0f015de60 2024-01-16T14:55:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.237+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f0154470 con 0x7fa0f015de60 2024-01-16T14:55:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.238+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0154660 con 0x7fa0f015de60 2024-01-16T14:55:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.239+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa0ec0075e0 con 0x7fa0f015de60 2024-01-16T14:55:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.239+0000 7fa0e27fc700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 0x7fa0d406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.239+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0ec088eb0 con 0x7fa0f015de60 2024-01-16T14:55:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.239+0000 7fa0f5ca1700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 0x7fa0d406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.240+0000 7fa0f5ca1700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 0x7fa0d406d840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009fd0 tx=0x7fa0e401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:52.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.244+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0ec053e70 con 0x7fa0f015de60 2024-01-16T14:55:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:52.504+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa0f0068670 con 0x7fa0f015de60 2024-01-16T14:55:52.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:55:52.943 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:55:53.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.426+0000 7fa0e27fc700 1 -- 172.21.15.26:0/4121549241 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa0ec0170c0 con 0x7fa0f015de60 2024-01-16T14:55:53.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 msgr2=0x7fa0d406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 0x7fa0d406d840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009fd0 tx=0x7fa0e401c040 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 msgr2=0x7fa0f01f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c400 tx=0x7fa0ec00c710 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 shutdown_connections 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015e240 0x7fa0f0158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f006c6f0 0x7fa0f0152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa0d406b380 0x7fa0d406d840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 --2- 172.21.15.26:0/4121549241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0f015de60 0x7fa0f01f3bd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 >> 172.21.15.26:0/4121549241 conn(0x7fa0f00b2830 msgr2=0x7fa0f00b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 shutdown_connections 2024-01-16T14:55:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.429+0000 7fa0f7f05700 1 -- 172.21.15.26:0/4121549241 wait complete. 2024-01-16T14:55:53.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-16T14:55:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.585+0000 7f1882b63700 1 Processor -- start 2024-01-16T14:55:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.585+0000 7f1882b63700 1 -- start start 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1882b63700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f187c15bde0 0x7f187c06ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1882b63700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f187c0b4240 0x7f187c15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1882b63700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c0cc670 con 0x7f187c15c1c0 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1882b63700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0cffb0 con 0x7f187c15bde0 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1882b63700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0cf7a0 con 0x7f187c0b4240 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f18808ff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f187c15bde0 0x7f187c06ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f18808ff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f187c15bde0 0x7f187c06ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41592/0 (socket says 172.21.15.26:41592) 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f187bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f187c15c1c0 0x7f187c0cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43978/0 (socket says 172.21.15.26:43978) 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f18808ff700 1 -- 172.21.15.26:0/3578406863 learned_addr learned my addr 172.21.15.26:0/3578406863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.586+0000 7f1881100700 1 --2- 172.21.15.26:0/3578406863 >> v2:172.21.15.93:3300/0 conn(0x7f187c0b4240 0x7f187c15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.587+0000 7f18808ff700 1 -- 172.21.15.26:0/3578406863 >> v2:172.21.15.93:3300/0 conn(0x7f187c0b4240 msgr2=0x7f187c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.587+0000 7f18808ff700 1 --2- 172.21.15.26:0/3578406863 >> v2:172.21.15.93:3300/0 conn(0x7f187c0b4240 0x7f187c15fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.587+0000 7f18808ff700 1 -- 172.21.15.26:0/3578406863 >> v1:172.21.15.26:6789/0 conn(0x7f187c15c1c0 legacy=0x7f187c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.587+0000 7f18808ff700 1 -- 172.21.15.26:0/3578406863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0cf9f0 con 0x7f187c15bde0 2024-01-16T14:55:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.587+0000 7f18808ff700 1 --2- 172.21.15.26:0/3578406863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c15bde0 0x7f187c06ece0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f187c0c8c50 tx=0x7f1870009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ecef718b0818ad server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.588+0000 7f187b7fe700 1 -- 172.21.15.26:0/3578406863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870011830 con 0x7f187c15bde0 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.588+0000 7f187b7fe700 1 -- 172.21.15.26:0/3578406863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1870011990 con 0x7f187c15bde0 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.588+0000 7f187b7fe700 1 -- 172.21.15.26:0/3578406863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870011d20 con 0x7f187c15bde0 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 -- 172.21.15.26:0/3578406863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c15bde0 msgr2=0x7f187c06ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 --2- 172.21.15.26:0/3578406863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c15bde0 0x7f187c06ece0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f187c0c8c50 tx=0x7f1870009bd0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 -- 172.21.15.26:0/3578406863 shutdown_connections 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 --2- 172.21.15.26:0/3578406863 >> v2:172.21.15.93:3300/0 conn(0x7f187c0b4240 0x7f187c15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 --2- 172.21.15.26:0/3578406863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c15bde0 0x7f187c06ece0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 -- 172.21.15.26:0/3578406863 >> 172.21.15.26:0/3578406863 conn(0x7f187c0b3720 msgr2=0x7f187c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 -- 172.21.15.26:0/3578406863 shutdown_connections 2024-01-16T14:55:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.589+0000 7f1882b63700 1 -- 172.21.15.26:0/3578406863 wait complete. 2024-01-16T14:55:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.590+0000 7f1882b63700 1 Processor -- start 2024-01-16T14:55:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.590+0000 7f1882b63700 1 -- start start 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f187c15bde0 0x7f187c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f187c154480 0x7f187c15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0ce280 con 0x7f187c0b4240 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0c4f30 con 0x7f187c154480 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1882b63700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0cdc20 con 0x7f187c15bde0 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f18808ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f18808ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41606/0 (socket says 172.21.15.26:41606) 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f18808ff700 1 -- 172.21.15.26:0/800573044 learned_addr learned my addr 172.21.15.26:0/800573044 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f1881100700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f187c154480 0x7f187c15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.591+0000 7f187bfff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f187c15bde0 0x7f187c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 -- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f187c15bde0 msgr2=0x7f187c0ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f187c15bde0 0x7f187c0ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 -- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f187c154480 msgr2=0x7f187c15e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f187c154480 0x7f187c15e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 -- 172.21.15.26:0/800573044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870009550 con 0x7f187c0b4240 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18808ff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1870000c00 tx=0x7f1870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1870004380 con 0x7f187c0b4240 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187001f070 con 0x7f187c0b4240 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c154920 con 0x7f187c0b4240 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18700036a0 con 0x7f187c0b4240 2024-01-16T14:55:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.592+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c0cdea0 con 0x7f187c0b4240 2024-01-16T14:55:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.593+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c0ce4f0 con 0x7f187c0b4240 2024-01-16T14:55:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.594+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1870021030 con 0x7f187c0b4240 2024-01-16T14:55:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.594+0000 7f18797fa700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 0x7f186006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.594+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f187008af30 con 0x7f187c0b4240 2024-01-16T14:55:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.594+0000 7f187bfff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 0x7f186006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.595+0000 7f187bfff700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 0x7f186006d810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f187c068930 tx=0x7f1868007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.599+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1870055ed0 con 0x7f187c0b4240 2024-01-16T14:55:53.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.839+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f187c0ccb80 con 0x7f186006b350 2024-01-16T14:55:53.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.841+0000 7f18797fa700 1 -- 172.21.15.26:0/800573044 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1868004af0 con 0x7f186006b350 2024-01-16T14:55:53.824 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:55:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:55:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-16T14:55:53.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:55:53.824 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:55:53.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 msgr2=0x7f186006d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 0x7f186006d810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f187c068930 tx=0x7f1868007ed0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 msgr2=0x7f187c0c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1870000c00 tx=0x7f1870004750 comp rx=0 tx=0).stop 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 shutdown_connections 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f187c154480 0x7f187c15e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f186006b350 0x7f186006d810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f187c0b4240 0x7f187c0c49f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 --2- 172.21.15.26:0/800573044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f187c15bde0 0x7f187c0ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 >> 172.21.15.26:0/800573044 conn(0x7f187c0b3720 msgr2=0x7f187c0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 shutdown_connections 2024-01-16T14:55:53.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.844+0000 7f1882b63700 1 -- 172.21.15.26:0/800573044 wait complete. 2024-01-16T14:55:53.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-16T14:55:53.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.997+0000 7f5ae73a1700 1 Processor -- start 2024-01-16T14:55:53.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.997+0000 7f5ae73a1700 1 -- start start 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae73a1700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5ae016d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae73a1700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00b4540 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae73a1700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00af570 con 0x7f5ae00b0610 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae73a1700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00af850 con 0x7f5ae0162590 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae513d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ae00c4ba0 0x7f5ae00c4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43980/0 (socket says 172.21.15.26:43980) 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.998+0000 7f5ae513d700 1 -- 172.21.15.26:0/88893702 learned_addr learned my addr 172.21.15.26:0/88893702 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ae593e700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5ae016d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682316086 0 0) 0x7f5ae00b4540 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5acc0035b0 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 537020232 0 0) 0x7f5acc0035b0 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 msgr2=0x7f5ae016d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:53.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5ae016d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 >> v1:172.21.15.93:6789/0 conn(0x7f5ae00b0610 legacy=0x7f5ae00aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00afb30 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f5adc001670 con 0x7f5ae00c4ba0 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 >> v1:172.21.15.26:6789/0 conn(0x7f5ae00c4ba0 legacy=0x7f5ae00c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5acc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc003f70 0x7f5acc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00af850 con 0x7f5acc003f70 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc00ba70 con 0x7f5acc004d90 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc00bc30 con 0x7f5ae0162590 2024-01-16T14:55:53.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ae493c700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc003f70 0x7f5acc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ae513d700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5acc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:53.999+0000 7f5ae593e700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 -- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 msgr2=0x7f5acc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ae0162590 0x7f5acc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 -- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5acc004d90 msgr2=0x7f5acc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5acc004d90 0x7f5acc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 -- 172.21.15.26:0/88893702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0035b0 con 0x7f5acc003f70 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ae493c700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc003f70 0x7f5acc004850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00018b0 tx=0x7f5acc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c0aab4f51402d server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0001370 con 0x7f5acc003f70 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad0012640 con 0x7f5acc003f70 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.000+0000 7f5ad7fff700 1 -- 172.21.15.26:0/88893702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00129f0 con 0x7f5acc003f70 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 -- 172.21.15.26:0/88893702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc003f70 msgr2=0x7f5acc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/88893702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5acc003f70 0x7f5acc004850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00018b0 tx=0x7f5acc005880 comp rx=0 tx=0).stop 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae593e700 1 -- 172.21.15.26:0/88893702 reap_dead start 2024-01-16T14:55:53.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 -- 172.21.15.26:0/88893702 shutdown_connections 2024-01-16T14:55:53.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 -- 172.21.15.26:0/88893702 >> 172.21.15.26:0/88893702 conn(0x7f5ae00b3720 msgr2=0x7f5ae00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:53.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 -- 172.21.15.26:0/88893702 shutdown_connections 2024-01-16T14:55:53.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.001+0000 7f5ae73a1700 1 -- 172.21.15.26:0/88893702 wait complete. 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.002+0000 7f5ae73a1700 1 Processor -- start 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.002+0000 7f5ae73a1700 1 -- start start 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5acc003f70 0x7f5ae00ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ae00b2530 0x7f5ae01f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00b29b0 con 0x7f5ae015d670 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae015daf0 con 0x7f5ae00b2530 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae73a1700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01f8c70 con 0x7f5acc003f70 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41620/0 (socket says 172.21.15.26:41620) 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 -- 172.21.15.26:0/1112289344 learned_addr learned my addr 172.21.15.26:0/1112289344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae513d700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5acc003f70 0x7f5ae00ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae593e700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ae00b2530 0x7f5ae01f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 -- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5acc003f70 msgr2=0x7f5ae00ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5acc003f70 0x7f5ae00ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 -- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ae00b2530 msgr2=0x7f5ae01f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ae00b2530 0x7f5ae01f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.003+0000 7f5ae493c700 1 -- 172.21.15.26:0/1112289344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00cdff0 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ae493c700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0014040 tx=0x7f5ad0001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0012bc0 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad0012d20 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae01f8990 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0006aa0 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.004+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae00cc580 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.005+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae006f620 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.006+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5ad0023070 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.006+0000 7f5ad5ffb700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 0x7f5ac406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.006+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ad008ac40 con 0x7f5ae015d670 2024-01-16T14:55:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.006+0000 7f5ae513d700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 0x7f5ac406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.007+0000 7f5ae513d700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 0x7f5ac406d810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000c00 tx=0x7f5adc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.011+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ad0055c00 con 0x7f5ae015d670 2024-01-16T14:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.219+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5ae0075480 con 0x7f5ae015d670 2024-01-16T14:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.220+0000 7f5ad5ffb700 1 -- 172.21.15.26:0/1112289344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ad002a070 con 0x7f5ae015d670 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 msgr2=0x7f5ac406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 0x7f5ac406d810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5adc000c00 tx=0x7f5adc009040 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 msgr2=0x7f5ae00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0014040 tx=0x7f5ad0001110 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 shutdown_connections 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ae00b2530 0x7f5ae01f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5acc003f70 0x7f5ae00ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f5ac406b350 0x7f5ac406d810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 --2- 172.21.15.26:0/1112289344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae015d670 0x7f5ae00b1ff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 >> 172.21.15.26:0/1112289344 conn(0x7f5ae00b3720 msgr2=0x7f5ae006d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 shutdown_connections 2024-01-16T14:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.222+0000 7f5ae73a1700 1 -- 172.21.15.26:0/1112289344 wait complete. 2024-01-16T14:55:54.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-16T14:55:54.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.376+0000 7fa8ff14c700 1 Processor -- start 2024-01-16T14:55:54.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.376+0000 7fa8ff14c700 1 -- start start 2024-01-16T14:55:54.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8ff14c700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa8f80b11d0 0x7fa8f80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8ff14c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8ff14c700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f806f110 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8ff14c700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f8161a10 con 0x7fa8f80b11d0 2024-01-16T14:55:54.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8ff14c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80d1010 con 0x7fa8f806fcb0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8f7fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fcee8700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa8f80b11d0 0x7fa8f80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fd6e9700 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fa8f80ceaf0 0x7fa8f806c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:39182/0 (socket says 172.21.15.26:39182) 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fd6e9700 1 -- 172.21.15.26:0/249381768 learned_addr learned my addr 172.21.15.26:0/249381768 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fcee8700 1 -- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 msgr2=0x7fa8f80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fcee8700 1 --2- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f80ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fcee8700 1 -- 172.21.15.26:0/249381768 >> v1:172.21.15.93:6789/0 conn(0x7fa8f80ceaf0 legacy=0x7fa8f806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8fcee8700 1 -- 172.21.15.26:0/249381768 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80cef00 con 0x7fa8f80b11d0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.377+0000 7fa8f77fe700 1 -- 172.21.15.26:0/249381768 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289206497 0 0) 0x7fa8f806f110 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8fcee8700 1 --2- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f80cfd60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009840 tx=0x7fa8e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86610166fbc47228 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8f77fe700 1 -- 172.21.15.26:0/249381768 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8011830 con 0x7fa8f80b11d0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8f77fe700 1 -- 172.21.15.26:0/249381768 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e8011990 con 0x7fa8f80b11d0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8f77fe700 1 -- 172.21.15.26:0/249381768 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8011d20 con 0x7fa8f80b11d0 2024-01-16T14:55:54.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 -- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 msgr2=0x7fa8f80cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f80cfd60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009840 tx=0x7fa8e8009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 -- 172.21.15.26:0/249381768 shutdown_connections 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f80cfd60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/249381768 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f80ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 -- 172.21.15.26:0/249381768 >> 172.21.15.26:0/249381768 conn(0x7fa8f80b3700 msgr2=0x7fa8f80b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 -- 172.21.15.26:0/249381768 shutdown_connections 2024-01-16T14:55:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.378+0000 7fa8ff14c700 1 -- 172.21.15.26:0/249381768 wait complete. 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.380+0000 7fa8ff14c700 1 Processor -- start 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.380+0000 7fa8ff14c700 1 -- start start 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f815dd00 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f815dfe0 con 0x7fa8f80b11d0 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8ff14c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f815e2c0 con 0x7fa8f806fcb0 2024-01-16T14:55:54.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55348/0 (socket says 172.21.15.26:55348) 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 -- 172.21.15.26:0/140778870 learned_addr learned my addr 172.21.15.26:0/140778870 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8f7fff700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fcee8700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 -- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 msgr2=0x7fa8f815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 -- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 msgr2=0x7fa8f815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 -- 172.21.15.26:0/140778870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80094a0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.381+0000 7fa8fd6e9700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000cba0 tx=0x7fa8f000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.382+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000aaa0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.382+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f000ac20 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.382+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f815e5a0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.382+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00070d0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.382+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f815a8e0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.383+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f806bc40 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.383+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa8f0007230 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.384+0000 7fa8f57fa700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 0x7fa8e406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.384+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8f00896a0 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.384+0000 7fa8fcee8700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 0x7fa8e406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.384+0000 7fa8fcee8700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 0x7fa8e406d810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80090b0 tx=0x7fa8e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.388+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8f0054470 con 0x7fa8f80ceaf0 2024-01-16T14:55:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.633+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8f80d09c0 con 0x7fa8e406b350 2024-01-16T14:55:54.621 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:55:54.640+0000 7f82346d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:55:54.621 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:55:54.640+0000 7f82346d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:55:54.621 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:55:54.640+0000 7f82346d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:55:54.621 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:55:54.640+0000 7f82346d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:55:54.621 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:55:54.640+0000 7f82346d6700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:55:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.674+0000 7fa8f57fa700 1 -- 172.21.15.26:0/140778870 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa8e8011c00 con 0x7fa8e406b350 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 msgr2=0x7fa8e406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 0x7fa8e406d810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80090b0 tx=0x7fa8e8008030 comp rx=0 tx=0).stop 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 msgr2=0x7fa8f81f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000cba0 tx=0x7fa8f000ceb0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 shutdown_connections 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fa8e406b350 0x7fa8e406d810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8f80ceaf0 0x7fa8f81f2d00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8f806fcb0 0x7fa8f815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 --2- 172.21.15.26:0/140778870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8f80b11d0 0x7fa8f815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 >> 172.21.15.26:0/140778870 conn(0x7fa8f80b3700 msgr2=0x7fa8f8161180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 shutdown_connections 2024-01-16T14:55:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.676+0000 7fa8ff14c700 1 -- 172.21.15.26:0/140778870 wait complete. 2024-01-16T14:55:54.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-16T14:55:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.831+0000 7f2a88281700 1 Processor -- start 2024-01-16T14:55:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.831+0000 7f2a88281700 1 -- start start 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a88281700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a801602e0 0x7f2a8015d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a88281700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 0x7f2a800ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a88281700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a800d0d00 con 0x7f2a8015ff00 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a88281700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a800cfc60 con 0x7f2a801602e0 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a88281700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800cff40 con 0x7f2a800cf0d0 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8581c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a801602e0 0x7f2a8015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8601d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a8015ff00 0x7f2a800cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54020/0 (socket says 172.21.15.26:54020) 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8601d700 1 -- 172.21.15.26:0/2474557352 learned_addr learned my addr 172.21.15.26:0/2474557352 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:55:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8681e700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 0x7f2a800ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8581c700 1 -- 172.21.15.26:0/2474557352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 msgr2=0x7f2a800ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8581c700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 0x7f2a800ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8581c700 1 -- 172.21.15.26:0/2474557352 >> v1:172.21.15.26:6789/0 conn(0x7f2a8015ff00 legacy=0x7f2a800cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8581c700 1 -- 172.21.15.26:0/2474557352 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800d0220 con 0x7f2a801602e0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8501b700 1 -- 172.21.15.26:0/2474557352 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909560755 0 0) 0x7f2a800d0d00 con 0x7f2a8015ff00 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.833+0000 7f2a8681e700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 0x7f2a800ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a8581c700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a801602e0 0x7f2a8015d140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009840 tx=0x7f2a70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833ff8c72cce10b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a8501b700 1 -- 172.21.15.26:0/2474557352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70011830 con 0x7f2a801602e0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a8501b700 1 -- 172.21.15.26:0/2474557352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70011990 con 0x7f2a801602e0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a8501b700 1 -- 172.21.15.26:0/2474557352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70011d20 con 0x7f2a801602e0 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a88281700 1 -- 172.21.15.26:0/2474557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a801602e0 msgr2=0x7f2a8015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a88281700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a801602e0 0x7f2a8015d140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009840 tx=0x7f2a70009b50 comp rx=0 tx=0).stop 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a88281700 1 -- 172.21.15.26:0/2474557352 shutdown_connections 2024-01-16T14:55:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.834+0000 7f2a88281700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a800cf0d0 0x7f2a800ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.835+0000 7f2a88281700 1 --2- 172.21.15.26:0/2474557352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a801602e0 0x7f2a8015d140 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.835+0000 7f2a88281700 1 -- 172.21.15.26:0/2474557352 >> 172.21.15.26:0/2474557352 conn(0x7f2a800b3800 msgr2=0x7f2a800b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.835+0000 7f2a88281700 1 -- 172.21.15.26:0/2474557352 shutdown_connections 2024-01-16T14:55:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.835+0000 7f2a88281700 1 -- 172.21.15.26:0/2474557352 wait complete. 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.836+0000 7f2a88281700 1 Processor -- start 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.836+0000 7f2a88281700 1 -- start start 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8015ff00 0x7f2a80150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a801602e0 0x7f2a800b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8006c8f0 con 0x7f2a800cf0d0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8006d6f0 con 0x7f2a801602e0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8006cae0 con 0x7f2a8015ff00 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55366/0 (socket says 172.21.15.26:55366) 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 -- 172.21.15.26:0/2583864509 learned_addr learned my addr 172.21.15.26:0/2583864509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8681e700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a801602e0 0x7f2a800b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8581c700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8015ff00 0x7f2a80150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 -- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8015ff00 msgr2=0x7f2a80150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8015ff00 0x7f2a80150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 -- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a801602e0 msgr2=0x7f2a800b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a801602e0 0x7f2a800b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 -- 172.21.15.26:0/2583864509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700094a0 con 0x7f2a800cf0d0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a8601d700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0011e0 tx=0x7f2a7c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c004c80 con 0x7f2a800cf0d0 2024-01-16T14:55:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c004e00 con 0x7f2a800cf0d0 2024-01-16T14:55:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8006cdc0 con 0x7f2a800cf0d0 2024-01-16T14:55:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.837+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007310 con 0x7f2a800cf0d0 2024-01-16T14:55:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.838+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8006d380 con 0x7f2a800cf0d0 2024-01-16T14:55:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.838+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a80075480 con 0x7f2a800cf0d0 2024-01-16T14:55:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.839+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2a7c007470 con 0x7f2a800cf0d0 2024-01-16T14:55:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.839+0000 7f2a76ffd700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 0x7f2a6406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:55:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.840+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c089a10 con 0x7f2a800cf0d0 2024-01-16T14:55:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.840+0000 7f2a8581c700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 0x7f2a6406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:55:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.840+0000 7f2a8581c700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 0x7f2a6406d840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a700090b0 tx=0x7f2a70008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:55:54.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:54.844+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a7c0547f0 con 0x7f2a800cf0d0 2024-01-16T14:55:55.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.092+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a800cf540 con 0x7f2a6406b380 2024-01-16T14:55:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.099+0000 7f2a76ffd700 1 -- 172.21.15.26:0/2583864509 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f2a70011b80 con 0x7f2a6406b380 2024-01-16T14:55:55.081 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 msgr2=0x7f2a6406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 0x7f2a6406d840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a700090b0 tx=0x7f2a70008030 comp rx=0 tx=0).stop 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 msgr2=0x7f2a8006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0011e0 tx=0x7f2a7c00cae0 comp rx=0 tx=0).stop 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 shutdown_connections 2024-01-16T14:55:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a801602e0 0x7f2a800b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f2a6406b380 0x7f2a6406d840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a800cf0d0 0x7f2a8006c3b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 --2- 172.21.15.26:0/2583864509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8015ff00 0x7f2a80150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.100+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 >> 172.21.15.26:0/2583864509 conn(0x7f2a800b3800 msgr2=0x7f2a8006ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.101+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 shutdown_connections 2024-01-16T14:55:55.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:55:55.101+0000 7f2a88281700 1 -- 172.21.15.26:0/2583864509 wait complete. 2024-01-16T14:55:55.096 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75 2024-01-16T14:55:55.097 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:55:55.097 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:55:55.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:55:55.116 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T14:56:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-16T14:56:01.221 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-16T14:56:01.221 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.03601 s, 178 MB/s 2024-01-16T14:56:01.222 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:56:01.222 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:56:01.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:01.244 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T14:56:07.297 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-16T14:56:07.298 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-16T14:56:07.298 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 5.98995 s, 179 MB/s 2024-01-16T14:56:07.298 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:56:07.298 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:56:07.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:07.319 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-16T14:56:13.403 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-16T14:56:13.403 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-16T14:56:13.403 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.01997 s, 178 MB/s 2024-01-16T14:56:13.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-16T14:56:13.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.565+0000 7f9e1edfe700 1 Processor -- start 2024-01-16T14:56:13.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.565+0000 7f9e1edfe700 1 -- start start 2024-01-16T14:56:13.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1edfe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9e180d0290 0x7f9e180d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1edfe700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9e180b6cf0 0x7f9e180b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1edfe700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180cffb0 con 0x7f9e1815bf70 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1edfe700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180d0c50 con 0x7f9e180d0290 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1edfe700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180b5b30 con 0x7f9e180b6cf0 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1cb9a700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9e1815bf70 0x7f9e180cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35188/0 (socket says 172.21.15.26:35188) 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1cb9a700 1 -- 172.21.15.26:0/3487096872 learned_addr learned my addr 172.21.15.26:0/3487096872 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e17fff700 1 --2- 172.21.15.26:0/3487096872 >> v2:172.21.15.26:3300/0 conn(0x7f9e180d0290 0x7f9e180d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.566+0000 7f9e1d39b700 1 --2- 172.21.15.26:0/3487096872 >> v2:172.21.15.93:3300/0 conn(0x7f9e180b6cf0 0x7f9e180b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e17fff700 1 -- 172.21.15.26:0/3487096872 >> v2:172.21.15.93:3300/0 conn(0x7f9e180b6cf0 msgr2=0x7f9e180b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e17fff700 1 --2- 172.21.15.26:0/3487096872 >> v2:172.21.15.93:3300/0 conn(0x7f9e180b6cf0 0x7f9e180b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e17fff700 1 -- 172.21.15.26:0/3487096872 >> v1:172.21.15.26:6789/0 conn(0x7f9e1815bf70 legacy=0x7f9e180cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e177fe700 1 -- 172.21.15.26:0/3487096872 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861900701 0 0) 0x7f9e180cffb0 con 0x7f9e1815bf70 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e17fff700 1 -- 172.21.15.26:0/3487096872 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180cca80 con 0x7f9e180d0290 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e17fff700 1 --2- 172.21.15.26:0/3487096872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e180d0290 0x7f9e180d0710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e180cf7b0 tx=0x7f9e04009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b4952d3a8bdb1e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e177fe700 1 -- 172.21.15.26:0/3487096872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04011830 con 0x7f9e180d0290 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e177fe700 1 -- 172.21.15.26:0/3487096872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e04011990 con 0x7f9e180d0290 2024-01-16T14:56:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.567+0000 7f9e177fe700 1 -- 172.21.15.26:0/3487096872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e04011d20 con 0x7f9e180d0290 2024-01-16T14:56:13.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 -- 172.21.15.26:0/3487096872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e180d0290 msgr2=0x7f9e180d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/3487096872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e180d0290 0x7f9e180d0710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e180cf7b0 tx=0x7f9e04009b20 comp rx=0 tx=0).stop 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 -- 172.21.15.26:0/3487096872 shutdown_connections 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/3487096872 >> v2:172.21.15.93:3300/0 conn(0x7f9e180b6cf0 0x7f9e180b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/3487096872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e180d0290 0x7f9e180d0710 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 -- 172.21.15.26:0/3487096872 >> 172.21.15.26:0/3487096872 conn(0x7f9e180d1cd0 msgr2=0x7f9e180d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 -- 172.21.15.26:0/3487096872 shutdown_connections 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.568+0000 7f9e1edfe700 1 -- 172.21.15.26:0/3487096872 wait complete. 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.569+0000 7f9e1edfe700 1 Processor -- start 2024-01-16T14:56:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.570+0000 7f9e1edfe700 1 -- start start 2024-01-16T14:56:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 0x7f9e1814fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9e180d0290 0x7f9e180c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c7350 con 0x7f9e1815bf70 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c7630 con 0x7f9e180b6cf0 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1edfe700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180c7910 con 0x7f9e180d0290 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48842/0 (socket says 172.21.15.26:48842) 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 -- 172.21.15.26:0/1033883803 learned_addr learned my addr 172.21.15.26:0/1033883803 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1cb9a700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 0x7f9e1814fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e17fff700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9e180d0290 0x7f9e180c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 -- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9e180d0290 msgr2=0x7f9e180c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9e180d0290 0x7f9e180c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 -- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 msgr2=0x7f9e1814fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 0x7f9e1814fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1d39b700 1 -- 172.21.15.26:0/1033883803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040094a0 con 0x7f9e1815bf70 2024-01-16T14:56:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.571+0000 7f9e1cb9a700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 0x7f9e1814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e1d39b700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000efc0 tx=0x7f9e1000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000cad0 con 0x7f9e1815bf70 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1000f040 con 0x7f9e1815bf70 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e180c7bf0 con 0x7f9e1815bf70 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000ce70 con 0x7f9e1815bf70 2024-01-16T14:56:13.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.572+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1815f160 con 0x7f9e1815bf70 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.573+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e180c8110 con 0x7f9e1815bf70 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.573+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f9e10014480 con 0x7f9e1815bf70 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.574+0000 7f9e157fa700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 0x7f9dfc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.574+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e1008a4a0 con 0x7f9e1815bf70 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.574+0000 7f9e1cb9a700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 0x7f9dfc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.574+0000 7f9e1cb9a700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 0x7f9dfc06d8a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c007fd0 tx=0x7f9e0c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.579+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e180c8110 con 0x7f9e1815bf70 2024-01-16T14:56:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.792+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f9e180c8110 con 0x7f9e1815bf70 2024-01-16T14:56:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.795+0000 7f9e157fa700 1 -- 172.21.15.26:0/1033883803 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v11)=0 v11) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9e180c8110 con 0x7f9e1815bf70 2024-01-16T14:56:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 msgr2=0x7f9dfc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 0x7f9dfc06d8a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c007fd0 tx=0x7f9e0c007de0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 msgr2=0x7f9e1815cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000efc0 tx=0x7f9e1000c0c0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 shutdown_connections 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9dfc06b3e0 0x7f9dfc06d8a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e1815bf70 0x7f9e1815cbd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e180b6cf0 0x7f9e1814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 --2- 172.21.15.26:0/1033883803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9e180d0290 0x7f9e180c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 >> 172.21.15.26:0/1033883803 conn(0x7f9e180d1cd0 msgr2=0x7f9e1806eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.797+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 shutdown_connections 2024-01-16T14:56:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.798+0000 7f9e1edfe700 1 -- 172.21.15.26:0/1033883803 wait complete. 2024-01-16T14:56:13.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-16T14:56:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.952+0000 7fd54fa90700 1 Processor -- start 2024-01-16T14:56:13.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 -- start start 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54806b620 0x7fd5480b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 0x7fd5480b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 0x7fd548161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd548071f50 con 0x7fd54806b620 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd548072230 con 0x7fd5480b5ab0 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.953+0000 7fd54fa90700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd548072510 con 0x7fd5481478c0 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54806b620 0x7fd5480b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54806b620 0x7fd5480b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48854/0 (socket says 172.21.15.26:48854) 2024-01-16T14:56:13.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 -- 172.21.15.26:0/1739434202 learned_addr learned my addr 172.21.15.26:0/1739434202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d02b700 1 --2- 172.21.15.26:0/1739434202 >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 0x7fd5480b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54e02d700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 0x7fd548161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 -- 172.21.15.26:0/1739434202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 msgr2=0x7fd548161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 0x7fd548161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 -- 172.21.15.26:0/1739434202 >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 msgr2=0x7fd5480b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 --2- 172.21.15.26:0/1739434202 >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 0x7fd5480b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 -- 172.21.15.26:0/1739434202 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480727f0 con 0x7fd54806b620 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d02b700 1 --2- 172.21.15.26:0/1739434202 >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 0x7fd5480b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54e02d700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 0x7fd548161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54d82c700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806b620 0x7fd5480b3a40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd54806d6f0 tx=0x7fd53c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831a256ec6b8af76 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54c82a700 1 -- 172.21.15.26:0/1739434202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00b870 con 0x7fd54806b620 2024-01-16T14:56:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54c82a700 1 -- 172.21.15.26:0/1739434202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c00b9d0 con 0x7fd54806b620 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.954+0000 7fd54c82a700 1 -- 172.21.15.26:0/1739434202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00bd60 con 0x7fd54806b620 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 -- 172.21.15.26:0/1739434202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806b620 msgr2=0x7fd5480b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806b620 0x7fd5480b3a40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd54806d6f0 tx=0x7fd53c009bd0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 -- 172.21.15.26:0/1739434202 shutdown_connections 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd5481478c0 0x7fd548161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 --2- 172.21.15.26:0/1739434202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54806b620 0x7fd5480b3a40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 --2- 172.21.15.26:0/1739434202 >> v2:172.21.15.93:3300/0 conn(0x7fd5480b5ab0 0x7fd5480b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 -- 172.21.15.26:0/1739434202 >> 172.21.15.26:0/1739434202 conn(0x7fd5480b2740 msgr2=0x7fd5480b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 -- 172.21.15.26:0/1739434202 shutdown_connections 2024-01-16T14:56:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.955+0000 7fd54fa90700 1 -- 172.21.15.26:0/1739434202 wait complete. 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.956+0000 7fd54fa90700 1 Processor -- start 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.956+0000 7fd54fa90700 1 -- start start 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd54806b620 0x7fd548145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5481478c0 0x7fd5480b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548145ea0 con 0x7fd5480b5ab0 2024-01-16T14:56:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481e3630 con 0x7fd5481478c0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54fa90700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481e3910 con 0x7fd54806b620 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54d02b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54d02b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48868/0 (socket says 172.21.15.26:48868) 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.957+0000 7fd54d02b700 1 -- 172.21.15.26:0/692149209 learned_addr learned my addr 172.21.15.26:0/692149209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d82c700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd54806b620 0x7fd548145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54e02d700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5481478c0 0x7fd5480b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 -- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd54806b620 msgr2=0x7fd548145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd54806b620 0x7fd548145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 -- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5481478c0 msgr2=0x7fd5480b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5481478c0 0x7fd5480b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 -- 172.21.15.26:0/692149209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c009550 con 0x7fd5480b5ab0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54d02b700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd54400d7d0 tx=0x7fd54400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544004030 con 0x7fd5480b5ab0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544004190 con 0x7fd5480b5ab0 2024-01-16T14:56:13.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5481e3bf0 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544004540 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.958+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5481e41b0 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.959+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd548068670 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.960+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7fd5440036a0 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.960+0000 7fd53a7fc700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 0x7fd52c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.960+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd54408a100 con 0x7fd5480b5ab0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.960+0000 7fd54d82c700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 0x7fd52c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:13.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.961+0000 7fd54d82c700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 0x7fd52c06d8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000c00 tx=0x7fd53c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:13.965+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd544055010 con 0x7fd5480b5ab0 2024-01-16T14:56:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.216+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd54806ba00 con 0x7fd52c06b3e0 2024-01-16T14:56:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.230+0000 7fd53a7fc700 1 -- 172.21.15.26:0/692149209 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd53c00bb40 con 0x7fd52c06b3e0 2024-01-16T14:56:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.232+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 msgr2=0x7fd52c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.232+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 0x7fd52c06d8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000c00 tx=0x7fd53c008030 comp rx=0 tx=0).stop 2024-01-16T14:56:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.232+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 msgr2=0x7fd5480b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.232+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd54400d7d0 tx=0x7fd54400db90 comp rx=0 tx=0).stop 2024-01-16T14:56:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 shutdown_connections 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd5481478c0 0x7fd5480b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd54806b620 0x7fd548145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd52c06b3e0 0x7fd52c06d8a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 --2- 172.21.15.26:0/692149209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5480b5ab0 0x7fd5480b1ad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 >> 172.21.15.26:0/692149209 conn(0x7fd5480b2740 msgr2=0x7fd5480afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 shutdown_connections 2024-01-16T14:56:14.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.233+0000 7fd54fa90700 1 -- 172.21.15.26:0/692149209 wait complete. 2024-01-16T14:56:14.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-16T14:56:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.387+0000 7fb8e5857700 1 Processor -- start 2024-01-16T14:56:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.387+0000 7fb8e5857700 1 -- start start 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8e5857700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8e015e200 0x7fb8e006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8e5857700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 0x7fb8e00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8e5857700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00d12b0 con 0x7fb8e00b3270 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8e5857700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e00d0980 con 0x7fb8e015e200 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8e5857700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00d0c60 con 0x7fb8e006c6e0 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8deffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8e015e200 0x7fb8e006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8deffd700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8e015e200 0x7fb8e006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33116/0 (socket says 172.21.15.26:33116) 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8deffd700 1 -- 172.21.15.26:0/3559125081 learned_addr learned my addr 172.21.15.26:0/3559125081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.388+0000 7fb8df7fe700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 0x7fb8e00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8deffd700 1 -- 172.21.15.26:0/3559125081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 msgr2=0x7fb8e00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8deffd700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 0x7fb8e00d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8deffd700 1 -- 172.21.15.26:0/3559125081 >> v1:172.21.15.93:6789/0 conn(0x7fb8e00b3270 legacy=0x7fb8e006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8deffd700 1 -- 172.21.15.26:0/3559125081 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00d0f40 con 0x7fb8e015e200 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8df7fe700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 0x7fb8e00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8ddffb700 1 -- 172.21.15.26:0/3559125081 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710125165 0 0) 0x7fb8e00d12b0 con 0x7fb8e00b3270 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8deffd700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e015e200 0x7fb8e006b9a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009920 tx=0x7fb8d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2f092d414cb84e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8ddffb700 1 -- 172.21.15.26:0/3559125081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0011830 con 0x7fb8e015e200 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8ddffb700 1 -- 172.21.15.26:0/3559125081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0011990 con 0x7fb8e015e200 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.389+0000 7fb8ddffb700 1 -- 172.21.15.26:0/3559125081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0011d20 con 0x7fb8e015e200 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 -- 172.21.15.26:0/3559125081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e015e200 msgr2=0x7fb8e006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e015e200 0x7fb8e006b9a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009920 tx=0x7fb8d0009c30 comp rx=0 tx=0).stop 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 -- 172.21.15.26:0/3559125081 shutdown_connections 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e006c6e0 0x7fb8e00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 --2- 172.21.15.26:0/3559125081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e015e200 0x7fb8e006b9a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 -- 172.21.15.26:0/3559125081 >> 172.21.15.26:0/3559125081 conn(0x7fb8e009d180 msgr2=0x7fb8e00d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 -- 172.21.15.26:0/3559125081 shutdown_connections 2024-01-16T14:56:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.390+0000 7fb8e5857700 1 -- 172.21.15.26:0/3559125081 wait complete. 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.391+0000 7fb8e5857700 1 Processor -- start 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.391+0000 7fb8e5857700 1 -- start start 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e00b3270 0x7fb8e00b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8e015e200 0x7fb8e0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e015d0a0 con 0x7fb8e006c6e0 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00cce50 con 0x7fb8e015e200 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8e5857700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00cd0d0 con 0x7fb8e00b3270 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8deffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33122/0 (socket says 172.21.15.26:33122) 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.392+0000 7fb8deffd700 1 -- 172.21.15.26:0/1061210975 learned_addr learned my addr 172.21.15.26:0/1061210975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8df7fe700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8e015e200 0x7fb8e0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8de7fc700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e00b3270 0x7fb8e00b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 -- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e00b3270 msgr2=0x7fb8e00b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e00b3270 0x7fb8e00b7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 -- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8e015e200 msgr2=0x7fb8e0151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8e015e200 0x7fb8e0151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 -- 172.21.15.26:0/1061210975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8009a70 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8deffd700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0000c00 tx=0x7fb8d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00036a0 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d001e070 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d0009550 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003a60 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.393+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e00cd690 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.394+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e0075480 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.395+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7fb8d0003bc0 con 0x7fb8e006c6e0 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.395+0000 7fb8c7fff700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 0x7fb8cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:14.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.395+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8d008aa80 con 0x7fb8e006c6e0 2024-01-16T14:56:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.395+0000 7fb8de7fc700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 0x7fb8cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.396+0000 7fb8de7fc700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 0x7fb8cc06d8a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b9f00 tx=0x7fb8d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:14.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.400+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8d0055990 con 0x7fb8e006c6e0 2024-01-16T14:56:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.648+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8e00b2fb0 con 0x7fb8cc06b3e0 2024-01-16T14:56:14.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.656+0000 7fb8c7fff700 1 -- 172.21.15.26:0/1061210975 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fb8d40050d0 con 0x7fb8cc06b3e0 2024-01-16T14:56:14.639 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75 2024-01-16T14:56:14.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 msgr2=0x7fb8cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 0x7fb8cc06d8a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b9f00 tx=0x7fb8d4009040 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 msgr2=0x7fb8e00b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0000c00 tx=0x7fb8d0004750 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 shutdown_connections 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8e015e200 0x7fb8e0151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb8cc06b3e0 0x7fb8cc06d8a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e006c6e0 0x7fb8e00b7970 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 --2- 172.21.15.26:0/1061210975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8e00b3270 0x7fb8e00b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 >> 172.21.15.26:0/1061210975 conn(0x7fb8e009d180 msgr2=0x7fb8e015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 shutdown_connections 2024-01-16T14:56:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:14.658+0000 7fb8e5857700 1 -- 172.21.15.26:0/1061210975 wait complete. 2024-01-16T14:56:14.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:14.655 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75) 2024-01-16T14:56:14.675 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75 2024-01-16T14:56:14.675 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3055190016" 2024-01-16T14:56:14.675 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:14.676 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 2024-01-16T14:56:16.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:16.677 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75) 2024-01-16T14:56:16.701 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/d6dbc299-fe9b-433b-9d73-cc0a991fab75 2024-01-16T14:56:16.702 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-16T14:56:16.702 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:16.702 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-16T14:56:16.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-16T14:56:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.911+0000 7ff760929700 1 Processor -- start 2024-01-16T14:56:16.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.911+0000 7ff760929700 1 -- start start 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.912+0000 7ff760929700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff75c15e0c0 0x7ff75c06ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.912+0000 7ff760929700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c15e4a0 con 0x7ff75c0b3f10 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff760929700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c161bc0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff760929700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0af5d0 con 0x7ff75c15e0c0 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff759d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff75c0b3f10 0x7ff75c06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50982/0 (socket says 172.21.15.26:50982) 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff759d9b700 1 -- 172.21.15.26:0/3972793107 learned_addr learned my addr 172.21.15.26:0/3972793107 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75a59c700 1 --2- 172.21.15.26:0/3972793107 >> v2:172.21.15.93:3300/0 conn(0x7ff75c15e0c0 0x7ff75c06ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674161700 0 0) 0x7ff74c001110 con 0x7ff75c0b3f10 2024-01-16T14:56:16.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7480035b0 con 0x7ff75c0b3f10 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140485569 0 0) 0x7ff75c161bc0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c001110 con 0x7ff75c06b6e0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75a59c700 1 -- 172.21.15.26:0/3972793107 >> v1:172.21.15.93:6789/0 conn(0x7ff75c06b6e0 legacy=0x7ff75c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75a59c700 1 -- 172.21.15.26:0/3972793107 >> v1:172.21.15.26:6789/0 conn(0x7ff75c0b3f10 legacy=0x7ff75c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75a59c700 1 -- 172.21.15.26:0/3972793107 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c162180 con 0x7ff75c15e0c0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.913+0000 7ff75a59c700 1 --2- 172.21.15.26:0/3972793107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c15e0c0 0x7ff75c06ffe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff744009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5ac50419116350 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011830 con 0x7ff75c15e0c0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744011990 con 0x7ff75c15e0c0 2024-01-16T14:56:16.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff75959a700 1 -- 172.21.15.26:0/3972793107 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011d20 con 0x7ff75c15e0c0 2024-01-16T14:56:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 -- 172.21.15.26:0/3972793107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c15e0c0 msgr2=0x7ff75c06ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 --2- 172.21.15.26:0/3972793107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c15e0c0 0x7ff75c06ffe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff744009b80 comp rx=0 tx=0).stop 2024-01-16T14:56:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 -- 172.21.15.26:0/3972793107 shutdown_connections 2024-01-16T14:56:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 --2- 172.21.15.26:0/3972793107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c15e0c0 0x7ff75c06ffe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:16.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 -- 172.21.15.26:0/3972793107 >> 172.21.15.26:0/3972793107 conn(0x7ff75c0b27b0 msgr2=0x7ff75c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 -- 172.21.15.26:0/3972793107 shutdown_connections 2024-01-16T14:56:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.914+0000 7ff760929700 1 -- 172.21.15.26:0/3972793107 wait complete. 2024-01-16T14:56:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.916+0000 7ff760929700 1 Processor -- start 2024-01-16T14:56:16.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.916+0000 7ff760929700 1 -- start start 2024-01-16T14:56:16.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c0b3f10 0x7ff75c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 0x7ff75c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c154460 con 0x7ff75c06b6e0 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c15e640 con 0x7ff75c0b3f10 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff760929700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1f83c0 con 0x7ff75c15e0c0 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33126/0 (socket says 172.21.15.26:33126) 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 -- 172.21.15.26:0/3305529156 learned_addr learned my addr 172.21.15.26:0/3305529156 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff759d9b700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c0b3f10 0x7ff75c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75ad9d700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 0x7ff75c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 -- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 msgr2=0x7ff75c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 0x7ff75c0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 -- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c0b3f10 msgr2=0x7ff75c1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c0b3f10 0x7ff75c1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:16.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 -- 172.21.15.26:0/3305529156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440094a0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75ad9d700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 0x7ff75c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.917+0000 7ff75a59c700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff744004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.918+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74401a9f0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.918+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74401ab50 con 0x7ff75c06b6e0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.918+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440243f0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.918+0000 7ff760929700 1 -- 172.21.15.26:0/3305529156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c150b10 con 0x7ff75c06b6e0 2024-01-16T14:56:16.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.918+0000 7ff760929700 1 -- 172.21.15.26:0/3305529156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c1f7e00 con 0x7ff75c06b6e0 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.919+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7340052f0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.919+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7ff74402d450 con 0x7ff75c06b6e0 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.920+0000 7ff752ffd700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 0x7ff73c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.920+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff74408afa0 con 0x7ff75c06b6e0 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.920+0000 7ff759d9b700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 0x7ff73c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:16.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.920+0000 7ff759d9b700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 0x7ff73c06d8a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff75c1f73f0 tx=0x7ff74c007e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:16.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:16.924+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff744055db0 con 0x7ff75c06b6e0 2024-01-16T14:56:17.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.180+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff734000c70 con 0x7ff73c06b3e0 2024-01-16T14:56:17.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.244+0000 7ff752ffd700 1 -- 172.21.15.26:0/3305529156 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff734000c70 con 0x7ff73c06b3e0 2024-01-16T14:56:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 msgr2=0x7ff73c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 0x7ff73c06d8a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff75c1f73f0 tx=0x7ff74c007e00 comp rx=0 tx=0).stop 2024-01-16T14:56:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 msgr2=0x7ff75c1f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff7440090b0 tx=0x7ff744004750 comp rx=0 tx=0).stop 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 shutdown_connections 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff75c15e0c0 0x7ff75c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff73c06b3e0 0x7ff73c06d8a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75c06b6e0 0x7ff75c1f7490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 --2- 172.21.15.26:0/3305529156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff75c0b3f10 0x7ff75c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 >> 172.21.15.26:0/3305529156 conn(0x7ff75c0b27b0 msgr2=0x7ff75c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 shutdown_connections 2024-01-16T14:56:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.247+0000 7ff750ff9700 1 -- 172.21.15.26:0/3305529156 wait complete. 2024-01-16T14:56:17.231 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.248+0000 7f8239ee1700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.232 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.248+0000 7f8239ee1700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.232 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.248+0000 7f8239ee1700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.232 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.248+0000 7f8239ee1700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.232 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.248+0000 7f8239ee1700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.245 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-16T14:56:17.245 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:17.246 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.263+0000 7f823a6e2700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.246 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.263+0000 7f823a6e2700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.246 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.263+0000 7f823a6e2700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.246 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.263+0000 7f823a6e2700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.246 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:17.263+0000 7f823a6e2700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.406+0000 7f6c1a053700 1 Processor -- start 2024-01-16T14:56:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.406+0000 7f6c1a053700 1 -- start start 2024-01-16T14:56:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c1a053700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 0x7f6c1406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c1a053700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c1406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c1a053700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c140cfd60 con 0x7f6c1415e000 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c1a053700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c140cf4b0 con 0x7f6c140d0ae0 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c1a053700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140cf1d0 con 0x7f6c140d0f60 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c137fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c1415e000 0x7f6c1406eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50992/0 (socket says 172.21.15.26:50992) 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.407+0000 7f6c137fe700 1 -- 172.21.15.26:0/410460106 learned_addr learned my addr 172.21.15.26:0/410460106 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c12ffd700 1 --2- 172.21.15.26:0/410460106 >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 0x7f6c1406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 --2- 172.21.15.26:0/410460106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c1406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c127fc700 1 -- 172.21.15.26:0/410460106 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271123752 0 0) 0x7f6c140cfd60 con 0x7f6c1415e000 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c127fc700 1 -- 172.21.15.26:0/410460106 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bfc0035b0 con 0x7f6c1415e000 2024-01-16T14:56:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 -- 172.21.15.26:0/410460106 >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 msgr2=0x7f6c1406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 --2- 172.21.15.26:0/410460106 >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 0x7f6c1406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 -- 172.21.15.26:0/410460106 >> v1:172.21.15.26:6789/0 conn(0x7f6c1415e000 legacy=0x7f6c1406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 -- 172.21.15.26:0/410460106 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c140d0000 con 0x7f6c140d0f60 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c12ffd700 1 --2- 172.21.15.26:0/410460106 >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 0x7f6c1406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c13fff700 1 --2- 172.21.15.26:0/410460106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c1406fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009d90 tx=0x7f6c0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf384635b237e33 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c127fc700 1 -- 172.21.15.26:0/410460106 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00e070 con 0x7f6c140d0f60 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c127fc700 1 -- 172.21.15.26:0/410460106 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c00bb20 con 0x7f6c140d0f60 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.408+0000 7f6c127fc700 1 -- 172.21.15.26:0/410460106 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c012430 con 0x7f6c140d0f60 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 -- 172.21.15.26:0/410460106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 msgr2=0x7f6c1406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 --2- 172.21.15.26:0/410460106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c1406fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009d90 tx=0x7f6c0c00b330 comp rx=0 tx=0).stop 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 -- 172.21.15.26:0/410460106 shutdown_connections 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 --2- 172.21.15.26:0/410460106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c1406fc30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 --2- 172.21.15.26:0/410460106 >> v2:172.21.15.93:3300/0 conn(0x7f6c140d0ae0 0x7f6c1406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 -- 172.21.15.26:0/410460106 >> 172.21.15.26:0/410460106 conn(0x7f6c140b3720 msgr2=0x7f6c140b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 -- 172.21.15.26:0/410460106 shutdown_connections 2024-01-16T14:56:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.409+0000 7f6c1a053700 1 -- 172.21.15.26:0/410460106 wait complete. 2024-01-16T14:56:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.410+0000 7f6c1a053700 1 Processor -- start 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 -- start start 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c140d0ae0 0x7f6c140c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c140b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1415e3e0 con 0x7f6c1415e000 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1415ef20 con 0x7f6c140d0ae0 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.411+0000 7f6c1a053700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1415e5f0 con 0x7f6c140d0f60 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33138/0 (socket says 172.21.15.26:33138) 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 -- 172.21.15.26:0/3955427785 learned_addr learned my addr 172.21.15.26:0/3955427785 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c137fe700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c140d0ae0 0x7f6c140c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c12ffd700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c140b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 -- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 msgr2=0x7f6c140b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c140b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 -- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c140d0ae0 msgr2=0x7f6c140c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c140d0ae0 0x7f6c140c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 -- 172.21.15.26:0/3955427785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080094a0 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c13fff700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cf310 tx=0x7f6c0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c012430 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c020070 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004280 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c009a70 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.412+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1415ebb0 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.413+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1415ed70 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.414+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f6c0c0043e0 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.415+0000 7f6bfbfff700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 0x7f6c0006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.415+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c0c022070 con 0x7f6c1415e000 2024-01-16T14:56:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.415+0000 7f6c137fe700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 0x7f6c0006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.415+0000 7f6c137fe700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 0x7f6c0006d8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c040011e0 tx=0x7f6c04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:17.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.419+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c0c056d70 con 0x7f6c1415e000 2024-01-16T14:56:17.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:17.674+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c140b4470 con 0x7f6c0006b3e0 2024-01-16T14:56:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:18.436+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f6c0c057140 con 0x7f6c1415e000 2024-01-16T14:56:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.315+0000 7f6bfbfff700 1 -- 172.21.15.26:0/3955427785 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f6c04007d00 con 0x7f6c0006b3e0 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.299 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 msgr2=0x7f6c0006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 0x7f6c0006d8a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c040011e0 tx=0x7f6c04009040 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 msgr2=0x7f6c140b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c140cf310 tx=0x7f6c0c004ad0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 shutdown_connections 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6c0006b3e0 0x7f6c0006d8a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1415e000 0x7f6c140b1560 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c140d0ae0 0x7f6c140c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 --2- 172.21.15.26:0/3955427785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c140d0f60 0x7f6c140b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 >> 172.21.15.26:0/3955427785 conn(0x7f6c140b3720 msgr2=0x7f6c140b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 shutdown_connections 2024-01-16T14:56:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.318+0000 7f6c1a053700 1 -- 172.21.15.26:0/3955427785 wait complete. 2024-01-16T14:56:20.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-16T14:56:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.475+0000 7fcd71e19700 1 Processor -- start 2024-01-16T14:56:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.475+0000 7fcd71e19700 1 -- start start 2024-01-16T14:56:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd71e19700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 0x7fcd6c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd71e19700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcd6c0c6060 0x7fcd6c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd71e19700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6c0c5bd0 con 0x7fcd6c0d0250 2024-01-16T14:56:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd71e19700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6c0cdb30 con 0x7fcd6c0c6060 2024-01-16T14:56:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd71e19700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0cd570 con 0x7fcd6c0cc780 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcd6c0d0250 0x7fcd6c0d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51008/0 (socket says 172.21.15.26:51008) 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6b7fe700 1 -- 172.21.15.26:0/3825381147 learned_addr learned my addr 172.21.15.26:0/3825381147 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6affd700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 0x7fcd6c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6bfff700 1 --2- 172.21.15.26:0/3825381147 >> v2:172.21.15.93:3300/0 conn(0x7fcd6c0c6060 0x7fcd6c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214576723 0 0) 0x7fcd6c0c5bd0 con 0x7fcd6c0d0250 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.476+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd540035b0 con 0x7fcd6c0d0250 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6bfff700 1 -- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 msgr2=0x7fcd6c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6bfff700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 0x7fcd6c0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6bfff700 1 -- 172.21.15.26:0/3825381147 >> v1:172.21.15.26:6789/0 conn(0x7fcd6c0d0250 legacy=0x7fcd6c0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398460058 0 0) 0x7fcd540035b0 con 0x7fcd6c0d0250 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6bfff700 1 -- 172.21.15.26:0/3825381147 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0cd850 con 0x7fcd6c0c6060 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6affd700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 0x7fcd6c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6bfff700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c0c64f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009d90 tx=0x7fcd6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48056b1d17621ddb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400e070 con 0x7fcd6c0c6060 2024-01-16T14:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64003a20 con 0x7fcd6c0c6060 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.477+0000 7fcd6a7fc700 1 -- 172.21.15.26:0/3825381147 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c001670 con 0x7fcd6c0c6060 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 -- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 msgr2=0x7fcd6c0c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c0c64f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009d90 tx=0x7fcd6400b330 comp rx=0 tx=0).stop 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 -- 172.21.15.26:0/3825381147 shutdown_connections 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c0c64f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 --2- 172.21.15.26:0/3825381147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0cc780 0x7fcd6c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 -- 172.21.15.26:0/3825381147 >> 172.21.15.26:0/3825381147 conn(0x7fcd6c0b3800 msgr2=0x7fcd6c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 -- 172.21.15.26:0/3825381147 shutdown_connections 2024-01-16T14:56:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.478+0000 7fcd71e19700 1 -- 172.21.15.26:0/3825381147 wait complete. 2024-01-16T14:56:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.479+0000 7fcd71e19700 1 Processor -- start 2024-01-16T14:56:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.479+0000 7fcd71e19700 1 -- start start 2024-01-16T14:56:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0d0250 0x7fcd6c06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c09fc40 con 0x7fcd6c0cc780 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1536f0 con 0x7fcd6c0c6060 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd71e19700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c09f680 con 0x7fcd6c0d0250 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd6affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33144/0 (socket says 172.21.15.26:33144) 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd6affd700 1 -- 172.21.15.26:0/2309230996 learned_addr learned my addr 172.21.15.26:0/2309230996 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd6b7fe700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.480+0000 7fcd6bfff700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0d0250 0x7fcd6c06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 -- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0d0250 msgr2=0x7fcd6c06c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0d0250 0x7fcd6c06c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 -- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 msgr2=0x7fcd6c1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 -- 172.21.15.26:0/2309230996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd600094a0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd6affd700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000ebe0 tx=0x7fcd6000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000c850 con 0x7fcd6c0cc780 2024-01-16T14:56:20.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000c9b0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd64009a70 con 0x7fcd6c0cc780 2024-01-16T14:56:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000cd40 con 0x7fcd6c0cc780 2024-01-16T14:56:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.481+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c0ca1d0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.482+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c0d14b0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.483+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fcd60013470 con 0x7fcd6c0cc780 2024-01-16T14:56:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.483+0000 7fcd53fff700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 0x7fcd5806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.483+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd6008a2d0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.483+0000 7fcd6b7fe700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 0x7fcd5806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.484+0000 7fcd6b7fe700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 0x7fcd5806d9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c000f60 tx=0x7fcd5c001e00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.488+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd60054e30 con 0x7fcd6c0cc780 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.784+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcd6c071560 con 0x7fcd6c0cc780 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.785+0000 7fcd53fff700 1 -- 172.21.15.26:0/2309230996 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+3293 (secure 0 0 0) 0x7fcd60017cf0 con 0x7fcd6c0cc780 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "d39fe2d6-b323-4e7f-9f15-004380a58c0d", 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-16T14:56:20.768 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 4, 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-16T14:56:20.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 113, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 32, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705416877, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705416870, 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-16T14:56:20.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 796, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222071177, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6668419072, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766425694208, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 85, 2024-01-16T14:56:20.771 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 178927341, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 46 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 18, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 2, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-16T14:56:20.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 4972 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-16T14:56:20.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 3, 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-16T14:55:39.362688+0000", 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "daemons": { 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "summary": "", 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "a": { 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "start_epoch": 0, 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "start_stamp": "0.000000", 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 0, 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": {}, 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: "task_status": {} 2024-01-16T14:56:20.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "b": { 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "start_epoch": 0, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "start_stamp": "0.000000", 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 0, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": {}, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "task_status": {} 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "c": { 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "start_epoch": 0, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "start_stamp": "0.000000", 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 0, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": {}, 2024-01-16T14:56:20.775 INFO:teuthology.orchestra.run.smithi026.stdout: "task_status": {} 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "d": { 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "start_epoch": 0, 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "start_stamp": "0.000000", 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 0, 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": {}, 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "task_status": {} 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": {} 2024-01-16T14:56:20.776 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 msgr2=0x7fcd5806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 0x7fcd5806d9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c000f60 tx=0x7fcd5c001e00 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 msgr2=0x7fcd6c09c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000ebe0 tx=0x7fcd6000eef0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 shutdown_connections 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd6c0d0250 0x7fcd6c06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd6c0c6060 0x7fcd6c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5806b530 0x7fcd5806d9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 --2- 172.21.15.26:0/2309230996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd6c0cc780 0x7fcd6c09c970 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 >> 172.21.15.26:0/2309230996 conn(0x7fcd6c0b3800 msgr2=0x7fcd6c0c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 shutdown_connections 2024-01-16T14:56:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.787+0000 7fcd71e19700 1 -- 172.21.15.26:0/2309230996 wait complete. 2024-01-16T14:56:20.784 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-16T14:56:20.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.940+0000 7f26727e2700 1 Processor -- start 2024-01-16T14:56:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.940+0000 7f26727e2700 1 -- start start 2024-01-16T14:56:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f26727e2700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f266c144c10 0x7f266c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f26727e2700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 0x7f266c06b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f26727e2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c06eb80 con 0x7f266c145100 2024-01-16T14:56:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f26727e2700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c06f500 con 0x7f266c144c10 2024-01-16T14:56:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f26727e2700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c0cd610 con 0x7f266c145650 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f266bfff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f266c144c10 0x7f266c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f266b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f266c145100 0x7f266c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51018/0 (socket says 172.21.15.26:51018) 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.941+0000 7f266b7fe700 1 -- 172.21.15.26:0/3144478789 learned_addr learned my addr 172.21.15.26:0/3144478789 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f2670d7f700 1 --2- 172.21.15.26:0/3144478789 >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 0x7f266c06b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266bfff700 1 -- 172.21.15.26:0/3144478789 >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 msgr2=0x7f266c06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266bfff700 1 --2- 172.21.15.26:0/3144478789 >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 0x7f266c06b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266bfff700 1 -- 172.21.15.26:0/3144478789 >> v1:172.21.15.26:6789/0 conn(0x7f266c145100 legacy=0x7f266c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266bfff700 1 -- 172.21.15.26:0/3144478789 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c160720 con 0x7f266c144c10 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266affd700 1 -- 172.21.15.26:0/3144478789 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110575485 0 0) 0x7f266c06eb80 con 0x7f266c145100 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f2670d7f700 1 --2- 172.21.15.26:0/3144478789 >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 0x7f266c06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266bfff700 1 --2- 172.21.15.26:0/3144478789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c144c10 0x7f266c06afd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f266c073800 tx=0x7f265c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f54144694b0a080 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266affd700 1 -- 172.21.15.26:0/3144478789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c011830 con 0x7f266c144c10 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266affd700 1 -- 172.21.15.26:0/3144478789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c011990 con 0x7f266c144c10 2024-01-16T14:56:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.942+0000 7f266affd700 1 -- 172.21.15.26:0/3144478789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c011d20 con 0x7f266c144c10 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 -- 172.21.15.26:0/3144478789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c144c10 msgr2=0x7f266c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 --2- 172.21.15.26:0/3144478789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c144c10 0x7f266c06afd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f266c073800 tx=0x7f265c009b20 comp rx=0 tx=0).stop 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 -- 172.21.15.26:0/3144478789 shutdown_connections 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 --2- 172.21.15.26:0/3144478789 >> v2:172.21.15.93:3300/0 conn(0x7f266c145650 0x7f266c06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 --2- 172.21.15.26:0/3144478789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c144c10 0x7f266c06afd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 -- 172.21.15.26:0/3144478789 >> 172.21.15.26:0/3144478789 conn(0x7f266c0b2740 msgr2=0x7f266c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 -- 172.21.15.26:0/3144478789 shutdown_connections 2024-01-16T14:56:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.943+0000 7f26727e2700 1 -- 172.21.15.26:0/3144478789 wait complete. 2024-01-16T14:56:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.944+0000 7f26727e2700 1 Processor -- start 2024-01-16T14:56:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.944+0000 7f26727e2700 1 -- start start 2024-01-16T14:56:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f266c144c10 0x7f266c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f266c145100 0x7f266c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c138490 con 0x7f266c145650 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0b4ef0 con 0x7f266c145100 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f26727e2700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0b51d0 con 0x7f266c144c10 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f2670d7f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f2670d7f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33172/0 (socket says 172.21.15.26:33172) 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f2670d7f700 1 -- 172.21.15.26:0/2553403147 learned_addr learned my addr 172.21.15.26:0/2553403147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f266bfff700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f266c144c10 0x7f266c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.945+0000 7f266b7fe700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f266c145100 0x7f266c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 -- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f266c144c10 msgr2=0x7f266c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f266c144c10 0x7f266c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 -- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f266c145100 msgr2=0x7f266c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f266c145100 0x7f266c0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 -- 172.21.15.26:0/2553403147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0094a0 con 0x7f266c145650 2024-01-16T14:56:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2670d7f700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2664007e60 tx=0x7f266400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400dd20 con 0x7f266c145650 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266400f040 con 0x7f266c145650 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0b54b0 con 0x7f266c145650 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664015640 con 0x7f266c145650 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.946+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c0b5a70 con 0x7f266c145650 2024-01-16T14:56:20.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.947+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0cd350 con 0x7f266c145650 2024-01-16T14:56:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.948+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f266401e3b0 con 0x7f266c145650 2024-01-16T14:56:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.948+0000 7f2668ff9700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 0x7f264c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.948+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2664089fa0 con 0x7f266c145650 2024-01-16T14:56:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.948+0000 7f266bfff700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 0x7f264c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.949+0000 7f266bfff700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 0x7f264c06d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f265c0090b0 tx=0x7f265c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:20.953+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26640bce90 con 0x7f266c145650 2024-01-16T14:56:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.219+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f266c1603b0 con 0x7f264c06b500 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.226+0000 7f2668ff9700 1 -- 172.21.15.26:0/2553403147 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f265c011630 con 0x7f264c06b500 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-16T14:56:21.210 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-16T14:56:21.211 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:21.211 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:21.211 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:21.211 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:21.211 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 msgr2=0x7f264c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 0x7f264c06d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f265c0090b0 tx=0x7f265c008030 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 msgr2=0x7f266c143430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2664007e60 tx=0x7f266400d3b0 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 shutdown_connections 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f264c06b500 0x7f264c06d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f266c145650 0x7f266c143430 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f266c144c10 0x7f266c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 --2- 172.21.15.26:0/2553403147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f266c145100 0x7f266c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 >> 172.21.15.26:0/2553403147 conn(0x7f266c0b2740 msgr2=0x7f266c0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 shutdown_connections 2024-01-16T14:56:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:21.229+0000 7f26727e2700 1 -- 172.21.15.26:0/2553403147 wait complete. 2024-01-16T14:56:21.226 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:22.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.401+0000 7f61fbcb6700 1 Processor -- start 2024-01-16T14:56:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.401+0000 7f61fbcb6700 1 -- start start 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fbcb6700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61f406b620 0x7f61f40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fbcb6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f41629c0 con 0x7f61f406ee60 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fbcb6700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f4071f50 con 0x7f61f40b5ab0 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fbcb6700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f4072230 con 0x7f61f406b620 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fa253700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61f406ee60 0x7f61f41622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51020/0 (socket says 172.21.15.26:51020) 2024-01-16T14:56:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61fa253700 1 -- 172.21.15.26:0/2855462014 learned_addr learned my addr 172.21.15.26:0/2855462014 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.402+0000 7f61f9a52700 1 --2- 172.21.15.26:0/2855462014 >> v2:172.21.15.26:3300/0 conn(0x7f61f406b620 0x7f61f40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f8a50700 1 -- 172.21.15.26:0/2855462014 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857205360 0 0) 0x7f61f41629c0 con 0x7f61f406ee60 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f8a50700 1 -- 172.21.15.26:0/2855462014 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e00035b0 con 0x7f61f406ee60 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f9a52700 1 -- 172.21.15.26:0/2855462014 >> v1:172.21.15.93:6789/0 conn(0x7f61f40b5ab0 legacy=0x7f61f4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f9a52700 1 -- 172.21.15.26:0/2855462014 >> v1:172.21.15.26:6789/0 conn(0x7f61f406ee60 legacy=0x7f61f41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f9a52700 1 -- 172.21.15.26:0/2855462014 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4072510 con 0x7f61f406b620 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f9a52700 1 --2- 172.21.15.26:0/2855462014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406b620 0x7f61f40b3a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009840 tx=0x7f61f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78bb2284a8fa6cb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f8a50700 1 -- 172.21.15.26:0/2855462014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0011830 con 0x7f61f406b620 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f8a50700 1 -- 172.21.15.26:0/2855462014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0011990 con 0x7f61f406b620 2024-01-16T14:56:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.403+0000 7f61f8a50700 1 -- 172.21.15.26:0/2855462014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f0011d20 con 0x7f61f406b620 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 -- 172.21.15.26:0/2855462014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406b620 msgr2=0x7f61f40b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/2855462014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406b620 0x7f61f40b3a40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009840 tx=0x7f61f0009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 -- 172.21.15.26:0/2855462014 shutdown_connections 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/2855462014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406b620 0x7f61f40b3a40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 -- 172.21.15.26:0/2855462014 >> 172.21.15.26:0/2855462014 conn(0x7f61f40b2740 msgr2=0x7f61f40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.404+0000 7f61fbcb6700 1 -- 172.21.15.26:0/2855462014 shutdown_connections 2024-01-16T14:56:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.405+0000 7f61fbcb6700 1 -- 172.21.15.26:0/2855462014 wait complete. 2024-01-16T14:56:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.407+0000 7f61fbcb6700 1 Processor -- start 2024-01-16T14:56:22.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.407+0000 7f61fbcb6700 1 -- start start 2024-01-16T14:56:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61f406b620 0x7f61f413c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61f40b5ab0 0x7f61f4142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f413eeb0 con 0x7f61f406ee60 2024-01-16T14:56:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f41404c0 con 0x7f61f40b5ab0 2024-01-16T14:56:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61fbcb6700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f413e510 con 0x7f61f406b620 2024-01-16T14:56:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61f9251700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61f9251700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33190/0 (socket says 172.21.15.26:33190) 2024-01-16T14:56:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61f9251700 1 -- 172.21.15.26:0/450741949 learned_addr learned my addr 172.21.15.26:0/450741949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.408+0000 7f61f9a52700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61f406b620 0x7f61f413c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 -- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61f406b620 msgr2=0x7f61f413c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61f406b620 0x7f61f413c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 -- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61f40b5ab0 msgr2=0x7f61f4142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61f40b5ab0 0x7f61f4142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 -- 172.21.15.26:0/450741949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00094a0 con 0x7f61f406ee60 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61f9251700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f61e400c4b0 tx=0x7f61e400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40052d0 con 0x7f61f406ee60 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4005430 con 0x7f61f406ee60 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f413e7f0 con 0x7f61f406ee60 2024-01-16T14:56:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40038c0 con 0x7f61f406ee60 2024-01-16T14:56:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.409+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f413f2d0 con 0x7f61f406ee60 2024-01-16T14:56:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.410+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f413eff0 con 0x7f61f406ee60 2024-01-16T14:56:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.411+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f61e4003a20 con 0x7f61f406ee60 2024-01-16T14:56:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.412+0000 7f61ea7fc700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 0x7f61d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.412+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61e40892d0 con 0x7f61f406ee60 2024-01-16T14:56:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.413+0000 7f61f9a52700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 0x7f61d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.415+0000 7f61f9a52700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 0x7f61d806d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f61f00090b0 tx=0x7f61f001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.419+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61f413eff0 con 0x7f61f406ee60 2024-01-16T14:56:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.787+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f61f4072ad0 con 0x7f61d806b500 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.799+0000 7f61ea7fc700 1 -- 172.21.15.26:0/450741949 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f61f0011b80 con 0x7f61d806b500 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:22.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 msgr2=0x7f61d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 0x7f61d806d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f61f00090b0 tx=0x7f61f001c040 comp rx=0 tx=0).stop 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 msgr2=0x7f61f41428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f61e400c4b0 tx=0x7f61e400c870 comp rx=0 tx=0).stop 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 shutdown_connections 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61f40b5ab0 0x7f61f4142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61f406b620 0x7f61f413c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f61d806b500 0x7f61d806d9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 --2- 172.21.15.26:0/450741949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61f406ee60 0x7f61f41428b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:22.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 >> 172.21.15.26:0/450741949 conn(0x7f61f40b2740 msgr2=0x7f61f40b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:22.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.805+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 shutdown_connections 2024-01-16T14:56:22.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:22.806+0000 7f61fbcb6700 1 -- 172.21.15.26:0/450741949 wait complete. 2024-01-16T14:56:22.814 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:23.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:56:23.010 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:56:23.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.997+0000 7f3931e3d700 1 Processor -- start 2024-01-16T14:56:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.997+0000 7f3931e3d700 1 -- start start 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f3931e3d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f392c15ff00 0x7f392c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f3931e3d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0cf610 con 0x7f392c15d140 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f3931e3d700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0d0d00 con 0x7f392c1602e0 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f3931e3d700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c15d570 con 0x7f392c15ff00 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f392c15ff00 0x7f392c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f392c15ff00 0x7f392c0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33200/0 (socket says 172.21.15.26:33200) 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f392c15d140 0x7f392c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51022/0 (socket says 172.21.15.26:51022) 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 -- 172.21.15.26:0/3698107589 learned_addr learned my addr 172.21.15.26:0/3698107589 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 -- 172.21.15.26:0/3698107589 >> v1:172.21.15.93:6789/0 conn(0x7f392c1602e0 legacy=0x7f392c0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 -- 172.21.15.26:0/3698107589 >> v1:172.21.15.26:6789/0 conn(0x7f392c15d140 legacy=0x7f392c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 -- 172.21.15.26:0/3698107589 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0cfc60 con 0x7f392c15ff00 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392a7fc700 1 -- 172.21.15.26:0/3698107589 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047763065 0 0) 0x7f392c0cf610 con 0x7f392c15d140 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392b7fe700 1 --2- 172.21.15.26:0/3698107589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c15ff00 0x7f392c0cf0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f391c0098f0 tx=0x7f391c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9315f02bbe405323 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392a7fc700 1 -- 172.21.15.26:0/3698107589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c00b8a0 con 0x7f392c15ff00 2024-01-16T14:56:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.998+0000 7f392a7fc700 1 -- 172.21.15.26:0/3698107589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c00ba00 con 0x7f392c15ff00 2024-01-16T14:56:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f392a7fc700 1 -- 172.21.15.26:0/3698107589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c00bd90 con 0x7f392c15ff00 2024-01-16T14:56:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 -- 172.21.15.26:0/3698107589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c15ff00 msgr2=0x7f392c0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 --2- 172.21.15.26:0/3698107589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c15ff00 0x7f392c0cf0d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f391c0098f0 tx=0x7f391c009c00 comp rx=0 tx=0).stop 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 -- 172.21.15.26:0/3698107589 shutdown_connections 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 --2- 172.21.15.26:0/3698107589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c15ff00 0x7f392c0cf0d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 -- 172.21.15.26:0/3698107589 >> 172.21.15.26:0/3698107589 conn(0x7f392c0b3800 msgr2=0x7f392c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 -- 172.21.15.26:0/3698107589 shutdown_connections 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:23.999+0000 7f3931e3d700 1 -- 172.21.15.26:0/3698107589 wait complete. 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.001+0000 7f3931e3d700 1 Processor -- start 2024-01-16T14:56:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.001+0000 7f3931e3d700 1 -- start start 2024-01-16T14:56:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.001+0000 7f3931e3d700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392c15d140 0x7f392c1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3931e3d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 0x7f392c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3931e3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 0x7f392c0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3931e3d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0aea70 con 0x7f392c1602e0 2024-01-16T14:56:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3931e3d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c06db20 con 0x7f392c15ff00 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3931e3d700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c158620 con 0x7f392c15d140 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 0x7f392c0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 0x7f392c0c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33212/0 (socket says 172.21.15.26:33212) 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 0x7f392c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392bfff700 1 -- 172.21.15.26:0/1690138080 learned_addr learned my addr 172.21.15.26:0/1690138080 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392b7fe700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392c15d140 0x7f392c1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 -- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392c15d140 msgr2=0x7f392c1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392c15d140 0x7f392c1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 -- 172.21.15.26:0/1690138080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 msgr2=0x7f392c0c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 0x7f392c0c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 -- 172.21.15.26:0/1690138080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c009550 con 0x7f392c15ff00 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f392affd700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 0x7f392c0c7770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3920001eb0 tx=0x7f392000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200052d0 con 0x7f392c15ff00 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920005430 con 0x7f392c15ff00 2024-01-16T14:56:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.002+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920003880 con 0x7f392c15ff00 2024-01-16T14:56:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.003+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c0c7e50 con 0x7f392c15ff00 2024-01-16T14:56:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.003+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c0c8410 con 0x7f392c15ff00 2024-01-16T14:56:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.004+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c0c8630 con 0x7f392c15ff00 2024-01-16T14:56:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.004+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f3920003bf0 con 0x7f392c15ff00 2024-01-16T14:56:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.004+0000 7f3913fff700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 0x7f391806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.005+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3920089280 con 0x7f392c15ff00 2024-01-16T14:56:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.005+0000 7f392b7fe700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 0x7f391806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.005+0000 7f392b7fe700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 0x7f391806d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f391c000c00 tx=0x7f391c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:23.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.009+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3920053e90 con 0x7f392c15ff00 2024-01-16T14:56:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.266+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f392c0d0500 con 0x7f391806b500 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.272+0000 7f3913fff700 1 -- 172.21.15.26:0/1690138080 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f391c00bb70 con 0x7f391806b500 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:24.257 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:24.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 msgr2=0x7f391806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 0x7f391806d9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f391c000c00 tx=0x7f391c008030 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 msgr2=0x7f392c0c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 0x7f392c0c7770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3920001eb0 tx=0x7f392000c6e0 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 shutdown_connections 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f391806b500 0x7f391806d9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f392c1602e0 0x7f392c0c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f392c15d140 0x7f392c1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 --2- 172.21.15.26:0/1690138080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f392c15ff00 0x7f392c0c7770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 >> 172.21.15.26:0/1690138080 conn(0x7f392c0b3800 msgr2=0x7f392c15bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 shutdown_connections 2024-01-16T14:56:24.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:24.275+0000 7f3931e3d700 1 -- 172.21.15.26:0/1690138080 wait complete. 2024-01-16T14:56:24.272 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:25.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.440+0000 7f9b4ed23700 1 Processor -- start 2024-01-16T14:56:25.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.440+0000 7f9b4ed23700 1 -- start start 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9b4815afc0 0x7f9b480730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4806c5d0 0x7f9b480b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b480b08f0 0x7f9b4806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b480b0d80 con 0x7f9b480b08f0 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4815f6b0 con 0x7f9b4815afc0 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4ed23700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4815eb40 con 0x7f9b4806c5d0 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4d2c0700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b480b08f0 0x7f9b4806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4d2c0700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b480b08f0 0x7f9b4806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36662/0 (socket says 172.21.15.26:36662) 2024-01-16T14:56:25.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4d2c0700 1 -- 172.21.15.26:0/4088179317 learned_addr learned my addr 172.21.15.26:0/4088179317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b47fff700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4806c5d0 0x7f9b480b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.441+0000 7f9b4cabf700 1 --2- 172.21.15.26:0/4088179317 >> v2:172.21.15.93:3300/0 conn(0x7f9b4815afc0 0x7f9b480730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 -- 172.21.15.26:0/4088179317 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4806c5d0 msgr2=0x7f9b480b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4806c5d0 0x7f9b480b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 -- 172.21.15.26:0/4088179317 >> v2:172.21.15.93:3300/0 conn(0x7f9b4815afc0 msgr2=0x7f9b480730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 --2- 172.21.15.26:0/4088179317 >> v2:172.21.15.93:3300/0 conn(0x7f9b4815afc0 0x7f9b480730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 -- 172.21.15.26:0/4088179317 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b480735f0 con 0x7f9b480b08f0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b4d2c0700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b4806afd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b4806fa80 tx=0x7f9b4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f85a7af8173b667 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b477fe700 1 -- 172.21.15.26:0/4088179317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000e070 con 0x7f9b480b08f0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b477fe700 1 -- 172.21.15.26:0/4088179317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4000b910 con 0x7f9b480b08f0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.442+0000 7f9b477fe700 1 -- 172.21.15.26:0/4088179317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000bcc0 con 0x7f9b480b08f0 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 -- 172.21.15.26:0/4088179317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 msgr2=0x7f9b4806afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b4806afd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9b4806fa80 tx=0x7f9b4000b330 comp rx=0 tx=0).stop 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 -- 172.21.15.26:0/4088179317 shutdown_connections 2024-01-16T14:56:25.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b4806afd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/4088179317 >> v2:172.21.15.93:3300/0 conn(0x7f9b4815afc0 0x7f9b480730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/4088179317 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4806c5d0 0x7f9b480b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 -- 172.21.15.26:0/4088179317 >> 172.21.15.26:0/4088179317 conn(0x7f9b480717f0 msgr2=0x7f9b480b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 -- 172.21.15.26:0/4088179317 shutdown_connections 2024-01-16T14:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.443+0000 7f9b4ed23700 1 -- 172.21.15.26:0/4088179317 wait complete. 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.444+0000 7f9b4ed23700 1 Processor -- start 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.444+0000 7f9b4ed23700 1 -- start start 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b4806c5d0 0x7f9b481f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4815afc0 0x7f9b48141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b481f6650 con 0x7f9b480b08f0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4814a980 con 0x7f9b4806c5d0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4ed23700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4814ac60 con 0x7f9b4815afc0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b47fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b47fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36666/0 (socket says 172.21.15.26:36666) 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b47fff700 1 -- 172.21.15.26:0/1658180051 learned_addr learned my addr 172.21.15.26:0/1658180051 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4d2c0700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4815afc0 0x7f9b48141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b4cabf700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b4806c5d0 0x7f9b481f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b47fff700 1 -- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4815afc0 msgr2=0x7f9b48141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.445+0000 7f9b47fff700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4815afc0 0x7f9b48141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b47fff700 1 -- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b4806c5d0 msgr2=0x7f9b481f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b47fff700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b4806c5d0 0x7f9b481f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b47fff700 1 -- 172.21.15.26:0/1658180051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40009b30 con 0x7f9b480b08f0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b47fff700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e930 tx=0x7f9b3c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c013680 con 0x7f9b480b08f0 2024-01-16T14:56:25.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c0137e0 con 0x7f9b480b08f0 2024-01-16T14:56:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4814af40 con 0x7f9b480b08f0 2024-01-16T14:56:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c013b70 con 0x7f9b480b08f0 2024-01-16T14:56:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.446+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4814b500 con 0x7f9b480b08f0 2024-01-16T14:56:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.447+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48075480 con 0x7f9b480b08f0 2024-01-16T14:56:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.448+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f9b3c013cd0 con 0x7f9b480b08f0 2024-01-16T14:56:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.448+0000 7f9b457fa700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 0x7f9b3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.448+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b3c089e30 con 0x7f9b480b08f0 2024-01-16T14:56:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.448+0000 7f9b4cabf700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 0x7f9b3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.449+0000 7f9b4cabf700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 0x7f9b3406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009b40 tx=0x7f9b38009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:25.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.453+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b3c054a20 con 0x7f9b480b08f0 2024-01-16T14:56:25.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.712+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b4815f340 con 0x7f9b3406b500 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.721+0000 7f9b457fa700 1 -- 172.21.15.26:0/1658180051 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f9b38011810 con 0x7f9b3406b500 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:25.703 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:25.704 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 msgr2=0x7f9b3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 0x7f9b3406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009b40 tx=0x7f9b38009210 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 msgr2=0x7f9b48148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e930 tx=0x7f9b3c00ec40 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 shutdown_connections 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b4815afc0 0x7f9b48141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b4806c5d0 0x7f9b481f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9b3406b500 0x7f9b3406d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 --2- 172.21.15.26:0/1658180051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b480b08f0 0x7f9b48148a50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:25.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.722+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 >> 172.21.15.26:0/1658180051 conn(0x7f9b480717f0 msgr2=0x7f9b4806d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.723+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 shutdown_connections 2024-01-16T14:56:25.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:25.723+0000 7f9b4ed23700 1 -- 172.21.15.26:0/1658180051 wait complete. 2024-01-16T14:56:25.720 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:26.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.887+0000 7f070daa3700 1 Processor -- start 2024-01-16T14:56:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.887+0000 7f070daa3700 1 -- start start 2024-01-16T14:56:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.887+0000 7f070daa3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080726a0 0x7f0708072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.887+0000 7f070daa3700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 0x7f0708072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f070daa3700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f070806bac0 0x7f070806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f070daa3700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080affd0 con 0x7f070806bac0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f070daa3700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070815f3e0 con 0x7f07080b3d90 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f070daa3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0708073500 con 0x7f07080726a0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f070806bac0 0x7f070806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f070806bac0 0x7f070806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36674/0 (socket says 172.21.15.26:36674) 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 -- 172.21.15.26:0/362847691 learned_addr learned my addr 172.21.15.26:0/362847691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0706ffd700 1 --2- 172.21.15.26:0/362847691 >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 0x7f0708072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f07077fe700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080726a0 0x7f0708072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 -- 172.21.15.26:0/362847691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080726a0 msgr2=0x7f0708072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080726a0 0x7f0708072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 -- 172.21.15.26:0/362847691 >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 msgr2=0x7f0708072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 --2- 172.21.15.26:0/362847691 >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 0x7f0708072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 -- 172.21.15.26:0/362847691 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070815e7d0 con 0x7f070806bac0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0706ffd700 1 --2- 172.21.15.26:0/362847691 >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 0x7f0708072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f0707fff700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f070806bf70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0708073740 tx=0x7f070000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72179a490ec7b160 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.888+0000 7f07067fc700 1 -- 172.21.15.26:0/362847691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000e070 con 0x7f070806bac0 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f07067fc700 1 -- 172.21.15.26:0/362847691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000b9c0 con 0x7f070806bac0 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f07067fc700 1 -- 172.21.15.26:0/362847691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000bd70 con 0x7f070806bac0 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 -- 172.21.15.26:0/362847691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 msgr2=0x7f070806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f070806bf70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0708073740 tx=0x7f070000b330 comp rx=0 tx=0).stop 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 -- 172.21.15.26:0/362847691 shutdown_connections 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f070806bf70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 --2- 172.21.15.26:0/362847691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080726a0 0x7f0708072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 --2- 172.21.15.26:0/362847691 >> v2:172.21.15.93:3300/0 conn(0x7f07080b3d90 0x7f0708072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 -- 172.21.15.26:0/362847691 >> 172.21.15.26:0/362847691 conn(0x7f07080717f0 msgr2=0x7f07080b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 -- 172.21.15.26:0/362847691 shutdown_connections 2024-01-16T14:56:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.889+0000 7f070daa3700 1 -- 172.21.15.26:0/362847691 wait complete. 2024-01-16T14:56:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.891+0000 7f070daa3700 1 Processor -- start 2024-01-16T14:56:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.891+0000 7f070daa3700 1 -- start start 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 0x7f0708149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080b3d90 0x7f070814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070814a8c0 con 0x7f070806bac0 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0708149cb0 con 0x7f07080726a0 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0708149f90 con 0x7f07080b3d90 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36688/0 (socket says 172.21.15.26:36688) 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 -- 172.21.15.26:0/1525086070 learned_addr learned my addr 172.21.15.26:0/1525086070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f0706ffd700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 0x7f0708149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f0707fff700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080b3d90 0x7f070814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 -- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080b3d90 msgr2=0x7f070814b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080b3d90 0x7f070814b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 -- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 msgr2=0x7f0708149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 0x7f0708149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:26.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 -- 172.21.15.26:0/1525086070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009b30 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f0706ffd700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 0x7f0708149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f07077fe700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e810 tx=0x7f06f800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8013610 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f8013770 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070814a270 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.892+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8013b00 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.893+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0708156550 con 0x7f070806bac0 2024-01-16T14:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.893+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0708075480 con 0x7f070806bac0 2024-01-16T14:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.894+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f06f8013c60 con 0x7f070806bac0 2024-01-16T14:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.894+0000 7f06effff700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 0x7f06f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.895+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06f8089cc0 con 0x7f070806bac0 2024-01-16T14:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.895+0000 7f0706ffd700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 0x7f06f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.895+0000 7f0706ffd700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 0x7f06f406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f07080afde0 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:26.899+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06f8090080 con 0x7f070806bac0 2024-01-16T14:56:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.154+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0708067010 con 0x7f06f406b500 2024-01-16T14:56:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.161+0000 7f06effff700 1 -- 172.21.15.26:0/1525086070 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f0708067010 con 0x7f06f406b500 2024-01-16T14:56:27.144 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:27.144 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:27.144 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:27.144 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:27.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:27.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:27.146 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 msgr2=0x7f06f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 0x7f06f406d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f07080afde0 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2024-01-16T14:56:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 msgr2=0x7f07081488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f06f800e810 tx=0x7f06f800ebd0 comp rx=0 tx=0).stop 2024-01-16T14:56:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 shutdown_connections 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07080b3d90 0x7f070814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f06f406b500 0x7f06f406d9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f070806bac0 0x7f07081488a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 --2- 172.21.15.26:0/1525086070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07080726a0 0x7f0708149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 >> 172.21.15.26:0/1525086070 conn(0x7f07080717f0 msgr2=0x7f07080ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 shutdown_connections 2024-01-16T14:56:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:27.163+0000 7f070daa3700 1 -- 172.21.15.26:0/1525086070 wait complete. 2024-01-16T14:56:27.160 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:28.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.384+0000 7fba0f656700 1 Processor -- start 2024-01-16T14:56:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.385+0000 7fba0f656700 1 -- start start 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.386+0000 7fba0f656700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fba0815e210 0x7fba080d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.386+0000 7fba0f656700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba080d2ae0 0x7fba080d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0f656700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080b73d0 con 0x7fba080cf520 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0f656700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba080b6df0 con 0x7fba080d2ae0 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0f656700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0815e5f0 con 0x7fba0815e210 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba080d2ae0 0x7fba080d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba080d2ae0 0x7fba080d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36700/0 (socket says 172.21.15.26:36700) 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 -- 172.21.15.26:0/3361186092 learned_addr learned my addr 172.21.15.26:0/3361186092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 -- 172.21.15.26:0/3361186092 >> v2:172.21.15.93:3300/0 conn(0x7fba0815e210 msgr2=0x7fba080d07d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 --2- 172.21.15.26:0/3361186092 >> v2:172.21.15.93:3300/0 conn(0x7fba0815e210 0x7fba080d07d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 -- 172.21.15.26:0/3361186092 >> v1:172.21.15.93:6789/0 conn(0x7fba080cf520 legacy=0x7fba080b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.387+0000 7fba0dbf3700 1 -- 172.21.15.26:0/3361186092 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080cfae0 con 0x7fba080d2ae0 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.388+0000 7fba0dbf3700 1 --2- 172.21.15.26:0/3361186092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080d0d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba0808c500 tx=0x7fba0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3065c9a813c98fc server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.388+0000 7fb9fffff700 1 -- 172.21.15.26:0/3361186092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000e070 con 0x7fba080d2ae0 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.388+0000 7fb9fffff700 1 -- 172.21.15.26:0/3361186092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0000b9c0 con 0x7fba080d2ae0 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.388+0000 7fb9fffff700 1 -- 172.21.15.26:0/3361186092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000bd70 con 0x7fba080d2ae0 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.389+0000 7fba0f656700 1 -- 172.21.15.26:0/3361186092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 msgr2=0x7fba080d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.389+0000 7fba0f656700 1 --2- 172.21.15.26:0/3361186092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080d0d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba0808c500 tx=0x7fba0000b330 comp rx=0 tx=0).stop 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.393+0000 7fba0f656700 1 -- 172.21.15.26:0/3361186092 shutdown_connections 2024-01-16T14:56:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.393+0000 7fba0f656700 1 --2- 172.21.15.26:0/3361186092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080d0d10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.393+0000 7fba0f656700 1 --2- 172.21.15.26:0/3361186092 >> v2:172.21.15.93:3300/0 conn(0x7fba0815e210 0x7fba080d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.393+0000 7fba0f656700 1 -- 172.21.15.26:0/3361186092 >> 172.21.15.26:0/3361186092 conn(0x7fba0809d180 msgr2=0x7fba080d3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:28.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.394+0000 7fba0f656700 1 -- 172.21.15.26:0/3361186092 shutdown_connections 2024-01-16T14:56:28.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.394+0000 7fba0f656700 1 -- 172.21.15.26:0/3361186092 wait complete. 2024-01-16T14:56:28.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.396+0000 7fba0f656700 1 Processor -- start 2024-01-16T14:56:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.396+0000 7fba0f656700 1 -- start start 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 0x7fba081f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba0815e210 0x7fba08158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba081f9160 con 0x7fba080d2ae0 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0815a400 con 0x7fba0815e210 2024-01-16T14:56:28.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.397+0000 7fba0f656700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0815a6e0 con 0x7fba080cf520 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36712/0 (socket says 172.21.15.26:36712) 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 -- 172.21.15.26:0/1250854707 learned_addr learned my addr 172.21.15.26:0/1250854707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0d3f2700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 0x7fba081f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 -- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 msgr2=0x7fba081f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 0x7fba081f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 -- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba0815e210 msgr2=0x7fba08158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba0815e210 0x7fba08158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0cbf1700 1 -- 172.21.15.26:0/1250854707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00009b30 con 0x7fba080d2ae0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.398+0000 7fba0d3f2700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 0x7fba081f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fba0cbf1700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c4e0 tx=0x7fb9f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80052d0 con 0x7fba080d2ae0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8005430 con 0x7fba080d2ae0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0815a9c0 con 0x7fba080d2ae0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80038c0 con 0x7fba080d2ae0 2024-01-16T14:56:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.399+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0815af80 con 0x7fba080d2ae0 2024-01-16T14:56:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.401+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba08075480 con 0x7fba080d2ae0 2024-01-16T14:56:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.401+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fb9f8003a20 con 0x7fba080d2ae0 2024-01-16T14:56:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.402+0000 7fb9fdffb700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 0x7fb9ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.402+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9f8089380 con 0x7fba080d2ae0 2024-01-16T14:56:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.404+0000 7fba0d3f2700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 0x7fb9ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.407+0000 7fba0d3f2700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 0x7fb9ec06d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba04009910 tx=0x7fba04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:28.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.411+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9f8053f70 con 0x7fba080d2ae0 2024-01-16T14:56:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.817+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fba080cfdc0 con 0x7fb9ec06b500 2024-01-16T14:56:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.823+0000 7fb9fdffb700 1 -- 172.21.15.26:0/1250854707 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fba04009de0 con 0x7fb9ec06b500 2024-01-16T14:56:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:28.805 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-16T14:56:28.806 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:28.807 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:28.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:28.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:28.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 msgr2=0x7fb9ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 0x7fb9ec06d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fba04009910 tx=0x7fba04008040 comp rx=0 tx=0).stop 2024-01-16T14:56:28.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 msgr2=0x7fba080ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:28.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c4e0 tx=0x7fb9f800c8a0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 shutdown_connections 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba0815e210 0x7fba08158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba080cf520 0x7fba081f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fb9ec06b500 0x7fb9ec06d9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 --2- 172.21.15.26:0/1250854707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba080d2ae0 0x7fba080ca760 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 >> 172.21.15.26:0/1250854707 conn(0x7fba0809d180 msgr2=0x7fba0806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 shutdown_connections 2024-01-16T14:56:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:28.826+0000 7fba0f656700 1 -- 172.21.15.26:0/1250854707 wait complete. 2024-01-16T14:56:28.822 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:29.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:29.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.990+0000 7f178a2e8700 1 Processor -- start 2024-01-16T14:56:29.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.990+0000 7f178a2e8700 1 -- start start 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f178a2e8700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f178415bde0 0x7f178415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f178a2e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f17840cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f178a2e8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840d0a90 con 0x7f17840cf2b0 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f178a2e8700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17840cefb0 con 0x7f178415bde0 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f178a2e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178406ee10 con 0x7f17840ce5c0 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f178415bde0 0x7f178415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f17837fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f17840cf2b0 0x7f17840d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45080/0 (socket says 172.21.15.26:45080) 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f17837fe700 1 -- 172.21.15.26:0/1981667109 learned_addr learned my addr 172.21.15.26:0/1981667109 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1788885700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f17840cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 -- 172.21.15.26:0/1981667109 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 msgr2=0x7f17840cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f17840cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 -- 172.21.15.26:0/1981667109 >> v1:172.21.15.26:6789/0 conn(0x7f17840cf2b0 legacy=0x7f17840d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1782ffd700 1 -- 172.21.15.26:0/1981667109 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449979295 0 0) 0x7f17840d0a90 con 0x7f17840cf2b0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 -- 172.21.15.26:0/1981667109 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178406f0f0 con 0x7f178415bde0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1788885700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f17840cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.991+0000 7f1783fff700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f178415c1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1774009840 tx=0x7f1774009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d82a90524436e7c server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f1782ffd700 1 -- 172.21.15.26:0/1981667109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774011830 con 0x7f178415bde0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f1782ffd700 1 -- 172.21.15.26:0/1981667109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1774011990 con 0x7f178415bde0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f1782ffd700 1 -- 172.21.15.26:0/1981667109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774011d20 con 0x7f178415bde0 2024-01-16T14:56:29.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 -- 172.21.15.26:0/1981667109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 msgr2=0x7f178415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f178415c1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1774009840 tx=0x7f1774009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 -- 172.21.15.26:0/1981667109 shutdown_connections 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f17840cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 --2- 172.21.15.26:0/1981667109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f178415c1c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 -- 172.21.15.26:0/1981667109 >> 172.21.15.26:0/1981667109 conn(0x7f17840b3720 msgr2=0x7f17840b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 -- 172.21.15.26:0/1981667109 shutdown_connections 2024-01-16T14:56:29.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.992+0000 7f178a2e8700 1 -- 172.21.15.26:0/1981667109 wait complete. 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.993+0000 7f178a2e8700 1 Processor -- start 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 -- start start 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f178409f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17840cf2b0 0x7f17840c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840b57e0 con 0x7f178415bde0 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178409f950 con 0x7f17840cf2b0 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.994+0000 7f178a2e8700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840b4bd0 con 0x7f17840ce5c0 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36724/0 (socket says 172.21.15.26:36724) 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 -- 172.21.15.26:0/3963423957 learned_addr learned my addr 172.21.15.26:0/3963423957 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f17837fe700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17840cf2b0 0x7f17840c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1783fff700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f178409f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 -- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 msgr2=0x7f178409f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f178409f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 -- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17840cf2b0 msgr2=0x7f17840c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17840cf2b0 0x7f17840c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 -- 172.21.15.26:0/3963423957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740094a0 con 0x7f178415bde0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1788885700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f177c00cef0 tx=0x7f177c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00ad00 con 0x7f178415bde0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c00ae60 con 0x7f178415bde0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c015680 con 0x7f178415bde0 2024-01-16T14:56:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17840b4eb0 con 0x7f178415bde0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.995+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840b5470 con 0x7f178415bde0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.996+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17840b5950 con 0x7f178415bde0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.997+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f177c00d430 con 0x7f178415bde0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.998+0000 7f1780ff9700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 0x7f177006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.998+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f177c08a390 con 0x7f178415bde0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.998+0000 7f1783fff700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 0x7f177006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:29.998+0000 7f1783fff700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 0x7f177006d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17740090b0 tx=0x7f177401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:29.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.003+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f177c054f80 con 0x7f178415bde0 2024-01-16T14:56:30.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.261+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f178406f6b0 con 0x7f177006b500 2024-01-16T14:56:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.268+0000 7f1780ff9700 1 -- 172.21.15.26:0/3963423957 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f1774011cb0 con 0x7f177006b500 2024-01-16T14:56:30.250 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-16T14:56:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-16T14:56:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:30.252 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 msgr2=0x7f177006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 0x7f177006d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17740090b0 tx=0x7f177401c040 comp rx=0 tx=0).stop 2024-01-16T14:56:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 msgr2=0x7f17840cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f177c00cef0 tx=0x7f177c00cf20 comp rx=0 tx=0).stop 2024-01-16T14:56:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 shutdown_connections 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f177006b500 0x7f177006d9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178415bde0 0x7f17840cb1a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17840ce5c0 0x7f178409f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 --2- 172.21.15.26:0/3963423957 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17840cf2b0 0x7f17840c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 >> 172.21.15.26:0/3963423957 conn(0x7f17840b3720 msgr2=0x7f17840b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 shutdown_connections 2024-01-16T14:56:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:30.271+0000 7f178a2e8700 1 -- 172.21.15.26:0/3963423957 wait complete. 2024-01-16T14:56:30.267 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:31.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.500+0000 7fc77af79700 1 Processor -- start 2024-01-16T14:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.501+0000 7fc77af79700 1 -- start start 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc774162770 0x7fc77406bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 0x7fc7740726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77406b9a0 con 0x7fc7740b17e0 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77406b070 con 0x7fc774162770 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.502+0000 7fc77af79700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77406b350 con 0x7fc77406c4a0 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:37682/0 (socket says 172.21.15.26:37682) 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 -- 172.21.15.26:0/1919561313 learned_addr learned my addr 172.21.15.26:0/1919561313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc779516700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 0x7fc7740726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 -- 172.21.15.26:0/1919561313 >> v2:172.21.15.93:3300/0 conn(0x7fc774162770 msgr2=0x7fc77406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.93:3300/0 conn(0x7fc774162770 0x7fc77406bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 -- 172.21.15.26:0/1919561313 >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 msgr2=0x7fc7740726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 0x7fc7740726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 -- 172.21.15.26:0/1919561313 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77406b630 con 0x7fc77406c4a0 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc779516700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 0x7fc7740726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc773fff700 1 --2- 172.21.15.26:0/1919561313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc774069520 tx=0x7fc760009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f11a418621d0ef server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.503+0000 7fc7737fe700 1 -- 172.21.15.26:0/1919561313 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000b870 con 0x7fc77406c4a0 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.504+0000 7fc7737fe700 1 -- 172.21.15.26:0/1919561313 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76000b9d0 con 0x7fc77406c4a0 2024-01-16T14:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.504+0000 7fc7737fe700 1 -- 172.21.15.26:0/1919561313 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76000bd60 con 0x7fc77406c4a0 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 -- 172.21.15.26:0/1919561313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 msgr2=0x7fc7740b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 --2- 172.21.15.26:0/1919561313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc774069520 tx=0x7fc760009bd0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 -- 172.21.15.26:0/1919561313 shutdown_connections 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.26:3300/0 conn(0x7fc7740b17e0 0x7fc7740726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 --2- 172.21.15.26:0/1919561313 >> v2:172.21.15.93:3300/0 conn(0x7fc774162770 0x7fc77406bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 --2- 172.21.15.26:0/1919561313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc77406c4a0 0x7fc7740b12a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 -- 172.21.15.26:0/1919561313 >> 172.21.15.26:0/1919561313 conn(0x7fc7740b5070 msgr2=0x7fc7740b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 -- 172.21.15.26:0/1919561313 shutdown_connections 2024-01-16T14:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.505+0000 7fc77af79700 1 -- 172.21.15.26:0/1919561313 wait complete. 2024-01-16T14:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.507+0000 7fc77af79700 1 Processor -- start 2024-01-16T14:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.508+0000 7fc77af79700 1 -- start start 2024-01-16T14:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 0x7fc77409df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc774162770 0x7fc7741f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774152e70 con 0x7fc7740b17e0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77409aff0 con 0x7fc77406c4a0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc77af79700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7741f8f60 con 0x7fc774162770 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc773fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc773fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36750/0 (socket says 172.21.15.26:36750) 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc773fff700 1 -- 172.21.15.26:0/3097989767 learned_addr learned my addr 172.21.15.26:0/3097989767 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc779516700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc774162770 0x7fc7741f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.509+0000 7fc778d15700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 0x7fc77409df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 -- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc774162770 msgr2=0x7fc7741f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc774162770 0x7fc7741f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 -- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 msgr2=0x7fc77409df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 0x7fc77409df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 -- 172.21.15.26:0/3097989767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760009550 con 0x7fc7740b17e0 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc778d15700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 0x7fc77409df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc773fff700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc760004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760004310 con 0x7fc7740b17e0 2024-01-16T14:56:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76001f070 con 0x7fc7740b17e0 2024-01-16T14:56:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77409ce80 con 0x7fc7740b17e0 2024-01-16T14:56:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77409d440 con 0x7fc7740b17e0 2024-01-16T14:56:31.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.510+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600036a0 con 0x7fc7740b17e0 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.511+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7540052f0 con 0x7fc7740b17e0 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.512+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fc760021030 con 0x7fc7740b17e0 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.513+0000 7fc7717fa700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 0x7fc75806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.513+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc76008aff0 con 0x7fc7740b17e0 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.513+0000 7fc778d15700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 0x7fc75806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:31.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.513+0000 7fc778d15700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 0x7fc75806d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:31.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.518+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc760055d90 con 0x7fc7740b17e0 2024-01-16T14:56:31.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.914+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc754000bc0 con 0x7fc75806b500 2024-01-16T14:56:31.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.933+0000 7fc7717fa700 1 -- 172.21.15.26:0/3097989767 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fc754000bc0 con 0x7fc75806b500 2024-01-16T14:56:31.915 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-16T14:56:31.916 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-16T14:56:31.917 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-16T14:56:31.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:31.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:31.917 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 msgr2=0x7fc75806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 0x7fc75806d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc768008040 comp rx=0 tx=0).stop 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 msgr2=0x7fc774152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc760000c00 tx=0x7fc760004750 comp rx=0 tx=0).stop 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 shutdown_connections 2024-01-16T14:56:31.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc774162770 0x7fc7741f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc77406c4a0 0x7fc77409df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fc75806b500 0x7fc75806d9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 --2- 172.21.15.26:0/3097989767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7740b17e0 0x7fc774152930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 >> 172.21.15.26:0/3097989767 conn(0x7fc7740b5070 msgr2=0x7fc7740b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 shutdown_connections 2024-01-16T14:56:31.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:31.936+0000 7fc77af79700 1 -- 172.21.15.26:0/3097989767 wait complete. 2024-01-16T14:56:31.933 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-16T14:56:32.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-16T14:56:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.103+0000 7f523a356700 1 Processor -- start 2024-01-16T14:56:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.104+0000 7f523a356700 1 -- start start 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f523a356700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 0x7f523415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f523a356700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 0x7f52340d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f523a356700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52340d0a90 con 0x7f52340ce5c0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f523a356700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523406ee10 con 0x7f523415bde0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f523a356700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523406f0f0 con 0x7f52340cf2b0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52388f3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52340ce5c0 0x7f52340cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45082/0 (socket says 172.21.15.26:45082) 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52388f3700 1 -- 172.21.15.26:0/3451423081 learned_addr learned my addr 172.21.15.26:0/3451423081 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52337fe700 1 --2- 172.21.15.26:0/3451423081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 0x7f52340d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f5233fff700 1 --2- 172.21.15.26:0/3451423081 >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 0x7f523415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f5232ffd700 1 -- 172.21.15.26:0/3451423081 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202710218 0 0) 0x7f52340d0a90 con 0x7f52340ce5c0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f5232ffd700 1 -- 172.21.15.26:0/3451423081 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f521c0035b0 con 0x7f52340ce5c0 2024-01-16T14:56:33.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52337fe700 1 -- 172.21.15.26:0/3451423081 >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 msgr2=0x7f523415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52337fe700 1 --2- 172.21.15.26:0/3451423081 >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 0x7f523415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52337fe700 1 -- 172.21.15.26:0/3451423081 >> v1:172.21.15.26:6789/0 conn(0x7f52340ce5c0 legacy=0x7f52340cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.105+0000 7f52337fe700 1 -- 172.21.15.26:0/3451423081 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523406f3d0 con 0x7f52340cf2b0 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f5233fff700 1 --2- 172.21.15.26:0/3451423081 >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 0x7f523415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f52337fe700 1 --2- 172.21.15.26:0/3451423081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 0x7f52340d01b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5228009840 tx=0x7f5228009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1752ff9e36cc58e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f5232ffd700 1 -- 172.21.15.26:0/3451423081 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228011830 con 0x7f52340cf2b0 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f5232ffd700 1 -- 172.21.15.26:0/3451423081 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228011990 con 0x7f52340cf2b0 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f5232ffd700 1 -- 172.21.15.26:0/3451423081 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228011d20 con 0x7f52340cf2b0 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f523a356700 1 -- 172.21.15.26:0/3451423081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 msgr2=0x7f52340d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f523a356700 1 --2- 172.21.15.26:0/3451423081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 0x7f52340d01b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5228009840 tx=0x7f5228009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.106+0000 7f523a356700 1 -- 172.21.15.26:0/3451423081 shutdown_connections 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.107+0000 7f523a356700 1 --2- 172.21.15.26:0/3451423081 >> v2:172.21.15.93:3300/0 conn(0x7f523415bde0 0x7f523415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.107+0000 7f523a356700 1 --2- 172.21.15.26:0/3451423081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340cf2b0 0x7f52340d01b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.107+0000 7f523a356700 1 -- 172.21.15.26:0/3451423081 >> 172.21.15.26:0/3451423081 conn(0x7f52340b3720 msgr2=0x7f52340b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.107+0000 7f523a356700 1 -- 172.21.15.26:0/3451423081 shutdown_connections 2024-01-16T14:56:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.107+0000 7f523a356700 1 -- 172.21.15.26:0/3451423081 wait complete. 2024-01-16T14:56:33.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.108+0000 7f523a356700 1 Processor -- start 2024-01-16T14:56:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.108+0000 7f523a356700 1 -- start start 2024-01-16T14:56:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 0x7f523415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f523415bde0 0x7f52340ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523415b4a0 con 0x7f52340cf2b0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340b2430 con 0x7f523415bde0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f523a356700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52340b2710 con 0x7f52340ce5c0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f5233fff700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 0x7f523415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36760/0 (socket says 172.21.15.26:36760) 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 -- 172.21.15.26:0/1897437735 learned_addr learned my addr 172.21.15.26:0/1897437735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52388f3700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f523415bde0 0x7f52340ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 -- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 msgr2=0x7f523415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 0x7f523415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 -- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f523415bde0 msgr2=0x7f52340ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f523415bde0 0x7f52340ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 -- 172.21.15.26:0/1897437735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240094a0 con 0x7f52340cf2b0 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f5233fff700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 0x7f523415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:33.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.109+0000 7f52337fe700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5228000c00 tx=0x7f5228004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.110+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280036a0 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.110+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522801f070 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.110+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52280094a0 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.110+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228003a60 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.110+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52341f2540 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.111+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52341f2380 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.112+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f5228003bc0 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.112+0000 7f5230ff9700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 0x7f522006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.112+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5228021070 con 0x7f52340cf2b0 2024-01-16T14:56:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.112+0000 7f5233fff700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 0x7f522006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.113+0000 7f5233fff700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 0x7f522006d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5224011380 tx=0x7f5224009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.117+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52280561e0 con 0x7f52340cf2b0 2024-01-16T14:56:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.377+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f52340aea00 con 0x7f522006b500 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.494+0000 7f5230ff9700 1 -- 172.21.15.26:0/1897437735 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f5224011d40 con 0x7f522006b500 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:33.477 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 msgr2=0x7f522006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 0x7f522006d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5224011380 tx=0x7f5224009210 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 msgr2=0x7f52340c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5228000c00 tx=0x7f5228004750 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 shutdown_connections 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f523415bde0 0x7f52340ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f52340ce5c0 0x7f523415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f522006b500 0x7f522006d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 --2- 172.21.15.26:0/1897437735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340cf2b0 0x7f52340c8360 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.497+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 >> 172.21.15.26:0/1897437735 conn(0x7f52340b3720 msgr2=0x7f52340cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.498+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 shutdown_connections 2024-01-16T14:56:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.498+0000 7f523a356700 1 -- 172.21.15.26:0/1897437735 wait complete. 2024-01-16T14:56:33.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-16T14:56:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.655+0000 7f9f6830a700 1 Processor -- start 2024-01-16T14:56:33.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.655+0000 7f9f6830a700 1 -- start start 2024-01-16T14:56:33.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.656+0000 7f9f6830a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.656+0000 7f9f6830a700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9f600ce560 0x7f9f600ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.656+0000 7f9f6830a700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6006f610 con 0x7f9f600cef20 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.656+0000 7f9f6830a700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600cf370 con 0x7f9f600ce560 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.656+0000 7f9f6830a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cd7f0 con 0x7f9f6015c010 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9f600ce560 0x7f9f600ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f660a6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9f600ce560 0x7f9f600ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.26:48266/0 (socket says 172.21.15.26:48266) 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f660a6700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.26:37728/0 (socket says 172.21.15.26:37728) 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 -- 172.21.15.26:0/899608650 learned_addr learned my addr 172.21.15.26:0/899608650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f668a7700 1 --1- 172.21.15.26:0/899608650 >> v1:172.21.15.93:6789/0 conn(0x7f9f600cef20 0x7f9f600cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.26:56174/0 (socket says 172.21.15.26:56174) 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 -- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 msgr2=0x7f9f600cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 -- 172.21.15.26:0/899608650 >> v1:172.21.15.93:6789/0 conn(0x7f9f600cef20 legacy=0x7f9f600cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 -- 172.21.15.26:0/899608650 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600cdac0 con 0x7f9f600ce560 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f660a6700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.657+0000 7f9f658a5700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f600ce560 0x7f9f600ce9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f500098f0 tx=0x7f9f50009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aefe9b3edf2c28 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f650a4700 1 -- 172.21.15.26:0/899608650 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50011830 con 0x7f9f600ce560 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f650a4700 1 -- 172.21.15.26:0/899608650 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50011990 con 0x7f9f600ce560 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f650a4700 1 -- 172.21.15.26:0/899608650 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50011d20 con 0x7f9f600ce560 2024-01-16T14:56:33.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 -- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f600ce560 msgr2=0x7f9f600ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f600ce560 0x7f9f600ce9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f500098f0 tx=0x7f9f50009c00 comp rx=0 tx=0).stop 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 -- 172.21.15.26:0/899608650 shutdown_connections 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f6015c010 0x7f9f600cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 --2- 172.21.15.26:0/899608650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f600ce560 0x7f9f600ce9e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 -- 172.21.15.26:0/899608650 >> 172.21.15.26:0/899608650 conn(0x7f9f600b3790 msgr2=0x7f9f600b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 -- 172.21.15.26:0/899608650 shutdown_connections 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.658+0000 7f9f6830a700 1 -- 172.21.15.26:0/899608650 wait complete. 2024-01-16T14:56:33.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.660+0000 7f9f6830a700 1 Processor -- start 2024-01-16T14:56:33.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.660+0000 7f9f6830a700 1 -- start start 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 0x7f9f6015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 0x7f9f600b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6015ed50 con 0x7f9f600ce560 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60150700 con 0x7f9f6015c010 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f6830a700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6015f1e0 con 0x7f9f600cef20 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f660a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f660a6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36764/0 (socket says 172.21.15.26:36764) 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f660a6700 1 -- 172.21.15.26:0/3607394171 learned_addr learned my addr 172.21.15.26:0/3607394171 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f668a7700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 0x7f9f600b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.661+0000 7f9f658a5700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 0x7f9f6015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 -- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 msgr2=0x7f9f6015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 0x7f9f6015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 -- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 msgr2=0x7f9f600b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 0x7f9f600b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 -- 172.21.15.26:0/3607394171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0094a0 con 0x7f9f600ce560 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f668a7700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 0x7f9f600b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f658a5700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 0x7f9f6015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f660a6700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00ebe0 tx=0x7f9f5c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00c8b0 con 0x7f9f600ce560 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00ca10 con 0x7f9f600ce560 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00cda0 con 0x7f9f600ce560 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50009550 con 0x7f9f600ce560 2024-01-16T14:56:33.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.662+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6015e420 con 0x7f9f600ce560 2024-01-16T14:56:33.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.663+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6015f030 con 0x7f9f600ce560 2024-01-16T14:56:33.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.664+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f9f5c013470 con 0x7f9f600ce560 2024-01-16T14:56:33.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.664+0000 7f9f56ffd700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 0x7f9f4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:33.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.664+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f5c08a040 con 0x7f9f600ce560 2024-01-16T14:56:33.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.665+0000 7f9f658a5700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 0x7f9f4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:33.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.665+0000 7f9f658a5700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 0x7f9f4406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009c40 tx=0x7f9f50008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:33.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.669+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f5c054bb0 con 0x7f9f600ce560 2024-01-16T14:56:33.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.911+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f6006f2a0 con 0x7f9f4406b500 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.913+0000 7f9f56ffd700 1 -- 172.21.15.26:0/3607394171 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9f50011c80 con 0x7f9f4406b500 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:33.895 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 msgr2=0x7f9f4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 0x7f9f4406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009c40 tx=0x7f9f50008030 comp rx=0 tx=0).stop 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 msgr2=0x7f9f601501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00ebe0 tx=0x7f9f5c009e20 comp rx=0 tx=0).stop 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 shutdown_connections 2024-01-16T14:56:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f6015c010 0x7f9f600b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f9f4406b500 0x7f9f4406d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f600ce560 0x7f9f601501c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 --2- 172.21.15.26:0/3607394171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f600cef20 0x7f9f6015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 >> 172.21.15.26:0/3607394171 conn(0x7f9f600b3790 msgr2=0x7f9f600b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 shutdown_connections 2024-01-16T14:56:33.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:33.915+0000 7f9f6830a700 1 -- 172.21.15.26:0/3607394171 wait complete. 2024-01-16T14:56:33.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-16T14:56:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.069+0000 7fd2dc855700 1 Processor -- start 2024-01-16T14:56:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.069+0000 7fd2dc855700 1 -- start start 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2d815afc0 0x7fd2d80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 0x7fd2d80b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 0x7fd2d806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80b0d80 con 0x7fd2d815afc0 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d815f6b0 con 0x7fd2d806c5d0 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2dc855700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d815eb40 con 0x7fd2d80b08f0 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2d815afc0 0x7fd2d80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2d815afc0 0x7fd2d80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36766/0 (socket says 172.21.15.26:36766) 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 -- 172.21.15.26:0/3772947087 learned_addr learned my addr 172.21.15.26:0/3772947087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d6d9d700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 0x7fd2d806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d5d9b700 1 --2- 172.21.15.26:0/3772947087 >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 0x7fd2d80b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 -- 172.21.15.26:0/3772947087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 msgr2=0x7fd2d806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 0x7fd2d806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 -- 172.21.15.26:0/3772947087 >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 msgr2=0x7fd2d80b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 --2- 172.21.15.26:0/3772947087 >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 0x7fd2d80b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d659c700 1 -- 172.21.15.26:0/3772947087 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80735f0 con 0x7fd2d815afc0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d6d9d700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 0x7fd2d806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.070+0000 7fd2d5d9b700 1 --2- 172.21.15.26:0/3772947087 >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 0x7fd2d80b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2d659c700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d815afc0 0x7fd2d80730b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80afe50 tx=0x7fd2c800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ff4003cd8e5dad server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2d559a700 1 -- 172.21.15.26:0/3772947087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800dd30 con 0x7fd2d815afc0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2d559a700 1 -- 172.21.15.26:0/3772947087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c800de90 con 0x7fd2d815afc0 2024-01-16T14:56:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2d559a700 1 -- 172.21.15.26:0/3772947087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8010690 con 0x7fd2d815afc0 2024-01-16T14:56:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 -- 172.21.15.26:0/3772947087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d815afc0 msgr2=0x7fd2d80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d815afc0 0x7fd2d80730b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80afe50 tx=0x7fd2c800be70 comp rx=0 tx=0).stop 2024-01-16T14:56:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 -- 172.21.15.26:0/3772947087 shutdown_connections 2024-01-16T14:56:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d80b08f0 0x7fd2d806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 --2- 172.21.15.26:0/3772947087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d815afc0 0x7fd2d80730b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 --2- 172.21.15.26:0/3772947087 >> v2:172.21.15.93:3300/0 conn(0x7fd2d806c5d0 0x7fd2d80b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.071+0000 7fd2dc855700 1 -- 172.21.15.26:0/3772947087 >> 172.21.15.26:0/3772947087 conn(0x7fd2d80717f0 msgr2=0x7fd2d80b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.072+0000 7fd2dc855700 1 -- 172.21.15.26:0/3772947087 shutdown_connections 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.072+0000 7fd2dc855700 1 -- 172.21.15.26:0/3772947087 wait complete. 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.073+0000 7fd2dc855700 1 Processor -- start 2024-01-16T14:56:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.073+0000 7fd2dc855700 1 -- start start 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d806c5d0 0x7fd2d81f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2d815afc0 0x7fd2d814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d809a6c0 con 0x7fd2d80b08f0 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81f78c0 con 0x7fd2d815afc0 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2dc855700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81f7ba0 con 0x7fd2d806c5d0 2024-01-16T14:56:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36778/0 (socket says 172.21.15.26:36778) 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 -- 172.21.15.26:0/2165114071 learned_addr learned my addr 172.21.15.26:0/2165114071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d6d9d700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2d815afc0 0x7fd2d814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d659c700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d806c5d0 0x7fd2d81f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 -- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d806c5d0 msgr2=0x7fd2d81f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d806c5d0 0x7fd2d81f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 -- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2d815afc0 msgr2=0x7fd2d814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2d815afc0 0x7fd2d814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 -- 172.21.15.26:0/2165114071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c800b7f0 con 0x7fd2d80b08f0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.074+0000 7fd2d5d9b700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d800 tx=0x7fd2cc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.075+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004030 con 0x7fd2d80b08f0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.075+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc004190 con 0x7fd2d80b08f0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.075+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d81f7e80 con 0x7fd2d80b08f0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.075+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004520 con 0x7fd2d80b08f0 2024-01-16T14:56:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.075+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d81f8160 con 0x7fd2d80b08f0 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.076+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d809cdb0 con 0x7fd2d80b08f0 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.076+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fd2cc0036a0 con 0x7fd2d80b08f0 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.077+0000 7fd2c6ffd700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 0x7fd2b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.077+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2cc089680 con 0x7fd2d80b08f0 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.077+0000 7fd2d659c700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 0x7fd2b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.077+0000 7fd2d659c700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 0x7fd2b806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800beb0 tx=0x7fd2c801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.082+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2d809cdb0 con 0x7fd2d80b08f0 2024-01-16T14:56:34.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.326+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2d815f340 con 0x7fd2b806b500 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.331+0000 7fd2c6ffd700 1 -- 172.21.15.26:0/2165114071 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7fd2c80040c0 con 0x7fd2b806b500 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-16T14:56:34.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:34.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1clone1" 2024-01-16T14:56:34.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:34.314 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 msgr2=0x7fd2b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 0x7fd2b806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800beb0 tx=0x7fd2c801c040 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 msgr2=0x7fd2d80af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00d800 tx=0x7fd2cc00db10 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 shutdown_connections 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd2d815afc0 0x7fd2d814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd2d806c5d0 0x7fd2d81f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fd2b806b500 0x7fd2b806d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 --2- 172.21.15.26:0/2165114071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2d80b08f0 0x7fd2d80af130 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 >> 172.21.15.26:0/2165114071 conn(0x7fd2d80717f0 msgr2=0x7fd2d815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.333+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 shutdown_connections 2024-01-16T14:56:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.334+0000 7fd2dc855700 1 -- 172.21.15.26:0/2165114071 wait complete. 2024-01-16T14:56:34.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-16T14:56:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.489+0000 7f6a93f82700 1 Processor -- start 2024-01-16T14:56:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.489+0000 7f6a93f82700 1 -- start start 2024-01-16T14:56:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a93f82700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 0x7f6a8c166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a93f82700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a93f82700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c06f160 con 0x7f6a8c162900 2024-01-16T14:56:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a93f82700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c0c5570 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a93f82700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c166e90 con 0x7f6a8c162520 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a9151d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a9151d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57354/0 (socket says 172.21.15.26:57354) 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a9151d700 1 -- 172.21.15.26:0/2480515524 learned_addr learned my addr 172.21.15.26:0/2480515524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.490+0000 7f6a91d1e700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 0x7f6a8c166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a9151d700 1 -- 172.21.15.26:0/2480515524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 msgr2=0x7f6a8c166950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a9151d700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 0x7f6a8c166950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a9151d700 1 -- 172.21.15.26:0/2480515524 >> v1:172.21.15.93:6789/0 conn(0x7f6a8c162900 legacy=0x7f6a8c162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a9151d700 1 -- 172.21.15.26:0/2480515524 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c06f440 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a91d1e700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 0x7f6a8c166950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a9151d700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098f0 tx=0x7f6a7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ba8cbdc5f2f980 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a90d1c700 1 -- 172.21.15.26:0/2480515524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00b8a0 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a90d1c700 1 -- 172.21.15.26:0/2480515524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c00ba00 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.491+0000 7f6a90d1c700 1 -- 172.21.15.26:0/2480515524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c00bd90 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 -- 172.21.15.26:0/2480515524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c0c4ba0 msgr2=0x7f6a8c0c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098f0 tx=0x7f6a7c009c00 comp rx=0 tx=0).stop 2024-01-16T14:56:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 -- 172.21.15.26:0/2480515524 shutdown_connections 2024-01-16T14:56:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162520 0x7f6a8c166950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 --2- 172.21.15.26:0/2480515524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c0c5030 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 -- 172.21.15.26:0/2480515524 >> 172.21.15.26:0/2480515524 conn(0x7f6a8c0b3700 msgr2=0x7f6a8c0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 -- 172.21.15.26:0/2480515524 shutdown_connections 2024-01-16T14:56:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.492+0000 7f6a93f82700 1 -- 172.21.15.26:0/2480515524 wait complete. 2024-01-16T14:56:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.493+0000 7f6a93f82700 1 Processor -- start 2024-01-16T14:56:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.493+0000 7f6a93f82700 1 -- start start 2024-01-16T14:56:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c1654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 0x7f6a8c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1f7bc0 con 0x7f6a8c162520 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0c7e60 con 0x7f6a8c0c4ba0 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a93f82700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0c9dd0 con 0x7f6a8c162900 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57366/0 (socket says 172.21.15.26:57366) 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 -- 172.21.15.26:0/404091447 learned_addr learned my addr 172.21.15.26:0/404091447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a91d1e700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c1654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9251f700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 0x7f6a8c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 -- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 msgr2=0x7f6a8c0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 0x7f6a8c0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 -- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6a8c0c4ba0 msgr2=0x7f6a8c1654e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c1654e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9151d700 1 -- 172.21.15.26:0/404091447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009550 con 0x7f6a8c162520 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.494+0000 7f6a9251f700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 0x7f6a8c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a9151d700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c004380 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c01f070 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0036a0 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c0ca0b0 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.495+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c0ca670 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.496+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c068670 con 0x7f6a8c162520 2024-01-16T14:56:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.496+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f6a7c021030 con 0x7f6a8c162520 2024-01-16T14:56:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.497+0000 7f6a827fc700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 0x7f6a7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.497+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a7c08b270 con 0x7f6a8c162520 2024-01-16T14:56:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.497+0000 7f6a91d1e700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 0x7f6a7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.497+0000 7f6a91d1e700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 0x7f6a7006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a880099d0 tx=0x7f6a88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.502+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a7c055e50 con 0x7f6a8c162520 2024-01-16T14:56:34.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.755+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a8c0aefa0 con 0x7f6a7006b500 2024-01-16T14:56:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.770+0000 7f6a827fc700 1 -- 172.21.15.26:0/404091447 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f6a88011750 con 0x7f6a7006b500 2024-01-16T14:56:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:34.752 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-16T14:56:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-16T14:56:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-16T14:56:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-16T14:56:34.753 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-16T14:56:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 msgr2=0x7f6a7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 0x7f6a7006d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a880099d0 tx=0x7f6a88008040 comp rx=0 tx=0).stop 2024-01-16T14:56:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 msgr2=0x7f6a8c1f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000c00 tx=0x7f6a7c004750 comp rx=0 tx=0).stop 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 shutdown_connections 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6a8c162900 0x7f6a8c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6a8c0c4ba0 0x7f6a8c1654e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6a7006b500 0x7f6a7006d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 --2- 172.21.15.26:0/404091447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a8c162520 0x7f6a8c1f7680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 >> 172.21.15.26:0/404091447 conn(0x7f6a8c0b3700 msgr2=0x7f6a8c0b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 shutdown_connections 2024-01-16T14:56:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.773+0000 7f6a93f82700 1 -- 172.21.15.26:0/404091447 wait complete. 2024-01-16T14:56:34.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-16T14:56:34.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.931+0000 7fcd76779700 1 Processor -- start 2024-01-16T14:56:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.931+0000 7fcd76779700 1 -- start start 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcd7015adf0 0x7fcd7006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd7006dd90 0x7fcd7006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd7015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7006e780 con 0x7fcd7006dd90 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd70073480 con 0x7fcd7015adf0 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd76779700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015b1d0 con 0x7fcd7015e900 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd6f7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd7006dd90 0x7fcd7006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd6f7fe700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd7006dd90 0x7fcd7006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57378/0 (socket says 172.21.15.26:57378) 2024-01-16T14:56:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd6f7fe700 1 -- 172.21.15.26:0/1358972586 learned_addr learned my addr 172.21.15.26:0/1358972586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd74d16700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd7015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.932+0000 7fcd6ffff700 1 --2- 172.21.15.26:0/1358972586 >> v2:172.21.15.93:3300/0 conn(0x7fcd7015adf0 0x7fcd7006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 -- 172.21.15.26:0/1358972586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 msgr2=0x7fcd7015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd7015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 -- 172.21.15.26:0/1358972586 >> v2:172.21.15.93:3300/0 conn(0x7fcd7015adf0 msgr2=0x7fcd7006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1358972586 >> v2:172.21.15.93:3300/0 conn(0x7fcd7015adf0 0x7fcd7006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 -- 172.21.15.26:0/1358972586 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70072870 con 0x7fcd7006dd90 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd7006e240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b4300 tx=0x7fcd64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b881d9a347ea5e36 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6effd700 1 -- 172.21.15.26:0/1358972586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400b870 con 0x7fcd7006dd90 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6effd700 1 -- 172.21.15.26:0/1358972586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6400b9d0 con 0x7fcd7006dd90 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd6effd700 1 -- 172.21.15.26:0/1358972586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6400bd60 con 0x7fcd7006dd90 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 -- 172.21.15.26:0/1358972586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 msgr2=0x7fcd7006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd7006e240 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b4300 tx=0x7fcd64009bd0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 -- 172.21.15.26:0/1358972586 shutdown_connections 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd7015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 --2- 172.21.15.26:0/1358972586 >> v2:172.21.15.93:3300/0 conn(0x7fcd7015adf0 0x7fcd7006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 --2- 172.21.15.26:0/1358972586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd7006e240 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.933+0000 7fcd76779700 1 -- 172.21.15.26:0/1358972586 >> 172.21.15.26:0/1358972586 conn(0x7fcd700b3e10 msgr2=0x7fcd700b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.934+0000 7fcd76779700 1 -- 172.21.15.26:0/1358972586 shutdown_connections 2024-01-16T14:56:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.934+0000 7fcd76779700 1 -- 172.21.15.26:0/1358972586 wait complete. 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.935+0000 7fcd76779700 1 Processor -- start 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.935+0000 7fcd76779700 1 -- start start 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd7015adf0 0x7fcd70159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd701599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70157ea0 con 0x7fcd7006dd90 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70158180 con 0x7fcd7015adf0 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd76779700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70158460 con 0x7fcd7015e900 2024-01-16T14:56:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57386/0 (socket says 172.21.15.26:57386) 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 -- 172.21.15.26:0/1077289343 learned_addr learned my addr 172.21.15.26:0/1077289343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd74d16700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd701599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd7015adf0 0x7fcd70159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 -- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 msgr2=0x7fcd701599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd701599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 -- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd7015adf0 msgr2=0x7fcd70159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd7015adf0 0x7fcd70159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 -- 172.21.15.26:0/1077289343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009550 con 0x7fcd7006dd90 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.936+0000 7fcd6ffff700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d800 tx=0x7fcd6000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.937+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600125f0 con 0x7fcd7006dd90 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.937+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60012750 con 0x7fcd7006dd90 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.937+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70158740 con 0x7fcd7006dd90 2024-01-16T14:56:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.937+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60012b00 con 0x7fcd7006dd90 2024-01-16T14:56:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.937+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70152b10 con 0x7fcd7006dd90 2024-01-16T14:56:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.938+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7006d890 con 0x7fcd7006dd90 2024-01-16T14:56:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.938+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fcd60012c60 con 0x7fcd7006dd90 2024-01-16T14:56:34.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.939+0000 7fcd6cff9700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 0x7fcd5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.939+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd6008a2e0 con 0x7fcd7006dd90 2024-01-16T14:56:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.939+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 0x7fcd5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.940+0000 7fcd6f7fe700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 0x7fcd5c06d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcd64014040 tx=0x7fcd64008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:34.944+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd60054e40 con 0x7fcd7006dd90 2024-01-16T14:56:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.196+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd70073110 con 0x7fcd5c06b500 2024-01-16T14:56:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.230+0000 7fcd6cff9700 1 -- 172.21.15.26:0/1077289343 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcd6400b670 con 0x7fcd5c06b500 2024-01-16T14:56:35.212 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.232+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 msgr2=0x7fcd5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 0x7fcd5c06d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcd64014040 tx=0x7fcd64008030 comp rx=0 tx=0).stop 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 msgr2=0x7fcd701525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000d800 tx=0x7fcd6000dbc0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 shutdown_connections 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd7015e900 0x7fcd701599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7fcd5c06b500 0x7fcd5c06d9c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd7006dd90 0x7fcd701525d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 --2- 172.21.15.26:0/1077289343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd7015adf0 0x7fcd70159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 >> 172.21.15.26:0/1077289343 conn(0x7fcd700b3e10 msgr2=0x7fcd700b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 shutdown_connections 2024-01-16T14:56:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.233+0000 7fcd76779700 1 -- 172.21.15.26:0/1077289343 wait complete. 2024-01-16T14:56:35.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-16T14:56:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.390+0000 7f9910808700 1 Processor -- start 2024-01-16T14:56:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.390+0000 7f9910808700 1 -- start start 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.391+0000 7f9910808700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.391+0000 7f9910808700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f990c0b3c60 0x7f990c0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.391+0000 7f9910808700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c15e990 con 0x7f990c158730 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.391+0000 7f9910808700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c072570 con 0x7f990c0b3c60 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.391+0000 7f9910808700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c15ec70 con 0x7f990c071bb0 2024-01-16T14:56:35.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f990c0b3c60 0x7f990c0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990a59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f990c158730 0x7f990c06f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38248/0 (socket says 172.21.15.26:38248) 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990a59c700 1 -- 172.21.15.26:0/2312366007 learned_addr learned my addr 172.21.15.26:0/2312366007 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f9909d9b700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 -- 172.21.15.26:0/2312366007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 msgr2=0x7f990c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 -- 172.21.15.26:0/2312366007 >> v1:172.21.15.26:6789/0 conn(0x7f990c158730 legacy=0x7f990c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 -- 172.21.15.26:0/2312366007 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c15ef50 con 0x7f990c0b3c60 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f9909d9b700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990ad9d700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c0b3c60 0x7f990c0b40b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f990400bfa0 tx=0x7f990400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e309b9ce9ae81c36 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990959a700 1 -- 172.21.15.26:0/2312366007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904010070 con 0x7f990c0b3c60 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990959a700 1 -- 172.21.15.26:0/2312366007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400b2e0 con 0x7f990c0b3c60 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.392+0000 7f990959a700 1 -- 172.21.15.26:0/2312366007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400dcf0 con 0x7f990c0b3c60 2024-01-16T14:56:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 -- 172.21.15.26:0/2312366007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c0b3c60 msgr2=0x7f990c0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c0b3c60 0x7f990c0b40b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f990400bfa0 tx=0x7f990400d330 comp rx=0 tx=0).stop 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 -- 172.21.15.26:0/2312366007 shutdown_connections 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c0b3c60 0x7f990c0b40b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 --2- 172.21.15.26:0/2312366007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 -- 172.21.15.26:0/2312366007 >> 172.21.15.26:0/2312366007 conn(0x7f990c0b27b0 msgr2=0x7f990c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 -- 172.21.15.26:0/2312366007 shutdown_connections 2024-01-16T14:56:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.393+0000 7f9910808700 1 -- 172.21.15.26:0/2312366007 wait complete. 2024-01-16T14:56:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.394+0000 7f9910808700 1 Processor -- start 2024-01-16T14:56:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 -- start start 2024-01-16T14:56:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 0x7f990c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c149c30 con 0x7f990c158730 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c14aca0 con 0x7f990c0b3c60 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f9910808700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c156b70 con 0x7f990c071bb0 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f990ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.395+0000 7f990ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57396/0 (socket says 172.21.15.26:57396) 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 -- 172.21.15.26:0/420958846 learned_addr learned my addr 172.21.15.26:0/420958846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9909d9b700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 0x7f990c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 -- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 msgr2=0x7f990c1496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 -- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 msgr2=0x7f990c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 0x7f990c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 -- 172.21.15.26:0/420958846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990400bc80 con 0x7f990c158730 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9909d9b700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 0x7f990c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-16T14:56:35.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f990ad9d700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b3570 tx=0x7f9904004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400dea0 con 0x7f990c158730 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400b450 con 0x7f990c158730 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c156e50 con 0x7f990c158730 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904009780 con 0x7f990c158730 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.396+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c0afce0 con 0x7f990c158730 2024-01-16T14:56:35.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.397+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c14e860 con 0x7f990c158730 2024-01-16T14:56:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.398+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f9904009290 con 0x7f990c158730 2024-01-16T14:56:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.398+0000 7f9902ffd700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 0x7f98ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.398+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f990408b970 con 0x7f990c158730 2024-01-16T14:56:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.399+0000 7f990a59c700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 0x7f98ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.399+0000 7f990a59c700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 0x7f98ec06d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0015d0 tx=0x7f98fc007ad0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.403+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9904056440 con 0x7f990c158730 2024-01-16T14:56:35.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.653+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f990c14ae10 con 0x7f98ec06b500 2024-01-16T14:56:35.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.671+0000 7f9902ffd700 1 -- 172.21.15.26:0/420958846 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f990c14ae10 con 0x7f98ec06b500 2024-01-16T14:56:35.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 msgr2=0x7f98ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 0x7f98ec06d9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0015d0 tx=0x7f98fc007ad0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 msgr2=0x7f990c14b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b3570 tx=0x7f9904004ad0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 shutdown_connections 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f98ec06b500 0x7f98ec06d9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990c158730 0x7f990c14b130 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f990c071bb0 0x7f990c1496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 --2- 172.21.15.26:0/420958846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f990c0b3c60 0x7f990c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 >> 172.21.15.26:0/420958846 conn(0x7f990c0b27b0 msgr2=0x7f990c0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 shutdown_connections 2024-01-16T14:56:35.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.674+0000 7f9910808700 1 -- 172.21.15.26:0/420958846 wait complete. 2024-01-16T14:56:35.658 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.675+0000 7f82376dc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.658 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.675+0000 7f82376dc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.658 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.675+0000 7f82376dc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.658 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.675+0000 7f82376dc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.658 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.675+0000 7f82376dc700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-16T14:56:35.676 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.694+0000 7f8235ed9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.676 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.694+0000 7f8235ed9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.676 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.694+0000 7f8235ed9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.676 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.694+0000 7f8235ed9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.676 INFO:tasks.ceph.mgr.x.smithi093.stderr:2024-01-16T14:56:35.694+0000 7f8235ed9700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-16T14:56:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.834+0000 7faf3112c700 1 Processor -- start 2024-01-16T14:56:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.834+0000 7faf3112c700 1 -- start start 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf3112c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 0x7faf2c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf3112c700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf2c0b0e90 0x7faf2c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf3112c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0d1230 con 0x7faf2c15e1c0 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf3112c700 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0d0c70 con 0x7faf2c0b0e90 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf3112c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0d0f50 con 0x7faf2c06fc50 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf2b59e700 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf2c0b0e90 0x7faf2c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf2ad9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf2c15e1c0 0x7faf2c06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38256/0 (socket says 172.21.15.26:38256) 2024-01-16T14:56:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.835+0000 7faf2ad9d700 1 -- 172.21.15.26:0/1563148604 learned_addr learned my addr 172.21.15.26:0/1563148604 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2a59c700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 0x7faf2c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2b59e700 1 -- 172.21.15.26:0/1563148604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 msgr2=0x7faf2c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2b59e700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 0x7faf2c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2b59e700 1 -- 172.21.15.26:0/1563148604 >> v1:172.21.15.26:6789/0 conn(0x7faf2c15e1c0 legacy=0x7faf2c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2b59e700 1 -- 172.21.15.26:0/1563148604 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c06c210 con 0x7faf2c0b0e90 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf29d9b700 1 -- 172.21.15.26:0/1563148604 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881188300 0 0) 0x7faf2c0d1230 con 0x7faf2c15e1c0 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2a59c700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 0x7faf2c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf2b59e700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c0b0e90 0x7faf2c06ca10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf24009840 tx=0x7faf24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237585e8637f14bb server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf29d9b700 1 -- 172.21.15.26:0/1563148604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24011830 con 0x7faf2c0b0e90 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf29d9b700 1 -- 172.21.15.26:0/1563148604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf24011990 con 0x7faf2c0b0e90 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.836+0000 7faf29d9b700 1 -- 172.21.15.26:0/1563148604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24011d20 con 0x7faf2c0b0e90 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 -- 172.21.15.26:0/1563148604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c0b0e90 msgr2=0x7faf2c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c0b0e90 0x7faf2c06ca10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf24009840 tx=0x7faf24009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 -- 172.21.15.26:0/1563148604 shutdown_connections 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c0b0e90 0x7faf2c06ca10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 --2- 172.21.15.26:0/1563148604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c06fc50 0x7faf2c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 -- 172.21.15.26:0/1563148604 >> 172.21.15.26:0/1563148604 conn(0x7faf2c0b3720 msgr2=0x7faf2c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 -- 172.21.15.26:0/1563148604 shutdown_connections 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.837+0000 7faf3112c700 1 -- 172.21.15.26:0/1563148604 wait complete. 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 Processor -- start 2024-01-16T14:56:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 -- start start 2024-01-16T14:56:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c0b0e90 0x7faf2c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf2c15e1c0 0x7faf2c0d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1f4ef0 con 0x7faf2c06fc50 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c09bc20 con 0x7faf2c15e1c0 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.839+0000 7faf3112c700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c09bf00 con 0x7faf2c0b0e90 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57420/0 (socket says 172.21.15.26:57420) 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 -- 172.21.15.26:0/1736307289 learned_addr learned my addr 172.21.15.26:0/1736307289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2a59c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c0b0e90 0x7faf2c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2b59e700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf2c15e1c0 0x7faf2c0d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 -- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c0b0e90 msgr2=0x7faf2c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c0b0e90 0x7faf2c0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 -- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf2c15e1c0 msgr2=0x7faf2c0d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf2c15e1c0 0x7faf2c0d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 -- 172.21.15.26:0/1736307289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240094a0 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf2ad9d700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00cc20 tx=0x7faf1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00ac20 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c00ad80 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c09c1e0 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.840+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c007310 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.841+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c15ad50 con 0x7faf2c06fc50 2024-01-16T14:56:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.842+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c09c700 con 0x7faf2c06fc50 2024-01-16T14:56:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.842+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7faf1c007470 con 0x7faf2c06fc50 2024-01-16T14:56:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.843+0000 7faf137fe700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 0x7faf1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.843+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5765+0+0 (secure 0 0 0) 0x7faf1c089d70 con 0x7faf2c06fc50 2024-01-16T14:56:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.843+0000 7faf2a59c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 0x7faf1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.843+0000 7faf2a59c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 0x7faf1806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf2c1621a0 tx=0x7faf20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:35.847+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf1c054920 con 0x7faf2c06fc50 2024-01-16T14:56:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.101+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7faf2c15e5a0 con 0x7faf1806b500 2024-01-16T14:56:36.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.120+0000 7faf137fe700 1 -- 172.21.15.26:0/1736307289 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faf2000c050 con 0x7faf1806b500 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 msgr2=0x7faf1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 0x7faf1806d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf2c1621a0 tx=0x7faf20008040 comp rx=0 tx=0).stop 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 msgr2=0x7faf2c15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faf1c00cc20 tx=0x7faf1c00a060 comp rx=0 tx=0).stop 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 shutdown_connections 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faf2c15e1c0 0x7faf2c0d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7faf1806b500 0x7faf1806d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf2c06fc50 0x7faf2c15a810 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 --2- 172.21.15.26:0/1736307289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faf2c0b0e90 0x7faf2c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 >> 172.21.15.26:0/1736307289 conn(0x7faf2c0b3720 msgr2=0x7faf2c0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:36.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 shutdown_connections 2024-01-16T14:56:36.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.122+0000 7faf3112c700 1 -- 172.21.15.26:0/1736307289 wait complete. 2024-01-16T14:56:36.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:36.119 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-16T14:56:36.160 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-16T14:56:36.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-16T14:56:36.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.365+0000 7f80da999700 1 Processor -- start 2024-01-16T14:56:36.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 -- start start 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80d40d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f80d40b6cf0 0x7f80d40b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d40cffb0 con 0x7f80d415bf70 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d40d0c50 con 0x7f80d40b6cf0 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.366+0000 7f80da999700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40b5b30 con 0x7f80d40d0290 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d3fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f80d415bf70 0x7f80d40cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38258/0 (socket says 172.21.15.26:38258) 2024-01-16T14:56:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d3fff700 1 -- 172.21.15.26:0/1849440681 learned_addr learned my addr 172.21.15.26:0/1849440681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d8f36700 1 --2- 172.21.15.26:0/1849440681 >> v2:172.21.15.93:3300/0 conn(0x7f80d40b6cf0 0x7f80d40b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940429943 0 0) 0x7f80d40cffb0 con 0x7f80d415bf70 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d37fe700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80d40d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80bc0035b0 con 0x7f80d415bf70 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88633426 0 0) 0x7f80bc0035b0 con 0x7f80d415bf70 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 msgr2=0x7f80d40d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80d40d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 >> v2:172.21.15.93:3300/0 conn(0x7f80d40b6cf0 msgr2=0x7f80d40b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 --2- 172.21.15.26:0/1849440681 >> v2:172.21.15.93:3300/0 conn(0x7f80d40b6cf0 0x7f80d40b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d40cca80 con 0x7f80d415bf70 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1819357777 0 0) 0x7f80c4001670 con 0x7f80d415bf70 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 >> v1:172.21.15.26:6789/0 conn(0x7f80d415bf70 legacy=0x7f80d40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80bc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80bc003f80 0x7f80bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80bc004d90 0x7f80bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40d0c50 con 0x7f80bc003f80 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40b5b30 con 0x7f80bc004d90 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80bc00ba10 con 0x7f80d40d0290 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.367+0000 7f80d37fe700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80bc003f80 0x7f80bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d3fff700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80bc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d8f36700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80bc004d90 0x7f80bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 -- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 msgr2=0x7f80bc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80d40d0290 0x7f80bc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 -- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80bc004d90 msgr2=0x7f80bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80bc004d90 0x7f80bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 -- 172.21.15.26:0/1849440681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0035b0 con 0x7f80bc003f80 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d37fe700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80bc003f80 0x7f80bc004850 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009b60 tx=0x7f80bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df130d593d7d9e server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c800e660 con 0x7f80bc003f80 2024-01-16T14:56:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c800e7c0 con 0x7f80bc003f80 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.368+0000 7f80d2ffd700 1 -- 172.21.15.26:0/1849440681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c800eb30 con 0x7f80bc003f80 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 -- 172.21.15.26:0/1849440681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80bc003f80 msgr2=0x7f80bc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 --2- 172.21.15.26:0/1849440681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80bc003f80 0x7f80bc004850 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009b60 tx=0x7f80bc005880 comp rx=0 tx=0).stop 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80d8f36700 1 -- 172.21.15.26:0/1849440681 reap_dead start 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 -- 172.21.15.26:0/1849440681 shutdown_connections 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 -- 172.21.15.26:0/1849440681 >> 172.21.15.26:0/1849440681 conn(0x7f80d40d1cd0 msgr2=0x7f80d40d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 -- 172.21.15.26:0/1849440681 shutdown_connections 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.369+0000 7f80da999700 1 -- 172.21.15.26:0/1849440681 wait complete. 2024-01-16T14:56:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.370+0000 7f80da999700 1 Processor -- start 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 -- start start 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 0x7f80d406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80d40c6700 0x7f80d40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40ca020 con 0x7f80d41512f0 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40c6580 con 0x7f80d40c6700 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80da999700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d40c70d0 con 0x7f80bc003f80 2024-01-16T14:56:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80d37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57440/0 (socket says 172.21.15.26:57440) 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.371+0000 7f80d37fe700 1 -- 172.21.15.26:0/2868926753 learned_addr learned my addr 172.21.15.26:0/2868926753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d8f36700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80d40c6700 0x7f80d40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d3fff700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 0x7f80d406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 -- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 msgr2=0x7f80d406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 0x7f80d406d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 -- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80d40c6700 msgr2=0x7f80d40c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80d40c6700 0x7f80d40c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 -- 172.21.15.26:0/2868926753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d406e110 con 0x7f80d41512f0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d3fff700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 0x7f80d406d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d37fe700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0046f0 tx=0x7f80c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c800e660 con 0x7f80d41512f0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8003ab0 con 0x7f80d41512f0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d406d870 con 0x7f80d41512f0 2024-01-16T14:56:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003e60 con 0x7f80d41512f0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.372+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d406de30 con 0x7f80d41512f0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.373+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d4075480 con 0x7f80d41512f0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.374+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f80c8013470 con 0x7f80d41512f0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.374+0000 7f80d0ff9700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 0x7f80c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.374+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f80c808b720 con 0x7f80d41512f0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.374+0000 7f80d3fff700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 0x7f80c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:36.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.375+0000 7f80d3fff700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 0x7f80c006d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.379+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80c8056480 con 0x7f80d41512f0 2024-01-16T14:56:36.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.623+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 --> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f80d4151770 con 0x7f80c006b500 2024-01-16T14:56:36.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:36.631+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 8) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f80c8056850 con 0x7f80d41512f0 2024-01-16T14:56:36.665 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-16T14:56:36.683+0000 7fcb0b119700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-16T14:56:36.717 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2024-01-16T14:56:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:38.703+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f80c805dee0 con 0x7f80d41512f0 2024-01-16T14:56:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.720+0000 7f80d0ff9700 1 -- 172.21.15.26:0/2868926753 <== mgr.4108 v2:172.21.15.93:6832/190365378 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f80c4007140 con 0x7f80c006b500 2024-01-16T14:56:39.703 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-16T14:56:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 msgr2=0x7f80c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 0x7f80c006d9c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4009040 comp rx=0 tx=0).stop 2024-01-16T14:56:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 msgr2=0x7f80d40c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0046f0 tx=0x7f80c80047c0 comp rx=0 tx=0).stop 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 shutdown_connections 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80d40c6700 0x7f80d40c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80bc003f80 0x7f80d406d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f80c006b500 0x7f80c006d9c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 --2- 172.21.15.26:0/2868926753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80d41512f0 0x7f80d40c6040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.724+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 >> 172.21.15.26:0/2868926753 conn(0x7f80d40d1cd0 msgr2=0x7f80d406f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.725+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 shutdown_connections 2024-01-16T14:56:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:39.725+0000 7f80da999700 1 -- 172.21.15.26:0/2868926753 wait complete. 2024-01-16T14:56:39.731 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-16T14:56:39.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T14:56:39.805 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-16T14:56:39.874 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-16T14:56:39.874 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:56:39.874 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:56:39.905 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-16T14:56:39.906 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-16T14:56:39.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-16T14:56:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.165+0000 7f1539102700 1 Processor -- start 2024-01-16T14:56:40.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.165+0000 7f1539102700 1 -- start start 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1539102700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 0x7f153406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1539102700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 0x7f153406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1539102700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15340cfe60 con 0x7f15340d0fd0 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1539102700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15340cf620 con 0x7f153415e330 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1539102700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340d06e0 con 0x7f15340d0b70 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153359e700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15340d0fd0 0x7f15340cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153359e700 1 -- 172.21.15.26:0/493428552 learned_addr learned my addr 172.21.15.26:0/493428552 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153259c700 1 --2- 172.21.15.26:0/493428552 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 0x7f153406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1532d9d700 1 --2- 172.21.15.26:0/493428552 >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 0x7f153406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1531d9b700 1 -- 172.21.15.26:0/493428552 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507735988 0 0) 0x7f15340cfe60 con 0x7f15340d0fd0 2024-01-16T14:56:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1531d9b700 1 -- 172.21.15.26:0/493428552 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151c0035b0 con 0x7f15340d0fd0 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153259c700 1 -- 172.21.15.26:0/493428552 >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 msgr2=0x7f153406ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153259c700 1 --2- 172.21.15.26:0/493428552 >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 0x7f153406ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153259c700 1 -- 172.21.15.26:0/493428552 >> v1:172.21.15.26:6789/0 conn(0x7f15340d0fd0 legacy=0x7f15340cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f153259c700 1 -- 172.21.15.26:0/493428552 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153415e710 con 0x7f15340d0b70 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.166+0000 7f1532d9d700 1 --2- 172.21.15.26:0/493428552 >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 0x7f153406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f153259c700 1 --2- 172.21.15.26:0/493428552 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 0x7f153406f720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1528009840 tx=0x7f1528009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c8963ce8de0f88 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1531d9b700 1 -- 172.21.15.26:0/493428552 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528011830 con 0x7f15340d0b70 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1531d9b700 1 -- 172.21.15.26:0/493428552 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528011990 con 0x7f15340d0b70 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1531d9b700 1 -- 172.21.15.26:0/493428552 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528011d20 con 0x7f15340d0b70 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 -- 172.21.15.26:0/493428552 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 msgr2=0x7f153406f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 --2- 172.21.15.26:0/493428552 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 0x7f153406f720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1528009840 tx=0x7f1528009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 -- 172.21.15.26:0/493428552 shutdown_connections 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 --2- 172.21.15.26:0/493428552 >> v2:172.21.15.93:3300/0 conn(0x7f153415e330 0x7f153406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 --2- 172.21.15.26:0/493428552 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0b70 0x7f153406f720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 -- 172.21.15.26:0/493428552 >> 172.21.15.26:0/493428552 conn(0x7f15340b3800 msgr2=0x7f15340b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:40.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 -- 172.21.15.26:0/493428552 shutdown_connections 2024-01-16T14:56:40.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.167+0000 7f1539102700 1 -- 172.21.15.26:0/493428552 wait complete. 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.169+0000 7f1539102700 1 Processor -- start 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.170+0000 7f1539102700 1 -- start start 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.170+0000 7f1539102700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15340d0b70 0x7f15340b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1539102700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 0x7f15340cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1539102700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1539102700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340ca800 con 0x7f153415e330 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1539102700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340b7590 con 0x7f15340d0b70 2024-01-16T14:56:40.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1539102700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340b7870 con 0x7f15340d0fd0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57444/0 (socket says 172.21.15.26:57444) 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153359e700 1 -- 172.21.15.26:0/555303600 learned_addr learned my addr 172.21.15.26:0/555303600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 0x7f15340cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f1532d9d700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15340d0b70 0x7f15340b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 -- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15340d0b70 msgr2=0x7f15340b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15340d0b70 0x7f15340b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 -- 172.21.15.26:0/555303600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 msgr2=0x7f153414fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 -- 172.21.15.26:0/555303600 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240094a0 con 0x7f15340d0fd0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153359e700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.171+0000 7f153259c700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 0x7f15340cb250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1528016000 tx=0x7f1528004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.172+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528004450 con 0x7f15340d0fd0 2024-01-16T14:56:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.172+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152801f070 con 0x7f15340d0fd0 2024-01-16T14:56:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.172+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280094a0 con 0x7f15340d0fd0 2024-01-16T14:56:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.172+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280038e0 con 0x7f15340d0fd0 2024-01-16T14:56:40.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.172+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15340cab00 con 0x7f15340d0fd0 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.173+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340caf70 con 0x7f15340d0fd0 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.173+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f1528003a40 con 0x7f15340d0fd0 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.173+0000 7f151b7fe700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 0x7f152006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.174+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1528021070 con 0x7f15340d0fd0 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.174+0000 7f1532d9d700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 0x7f152006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.174+0000 7f1532d9d700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 0x7f152006da50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1524009cd0 tx=0x7f1524009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.179+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15340caf70 con 0x7f15340d0fd0 2024-01-16T14:56:40.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.391+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15340caf70 con 0x7f15340d0fd0 2024-01-16T14:56:40.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.392+0000 7f151b7fe700 1 -- 172.21.15.26:0/555303600 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12967 (secure 0 0 0) 0x7f15340caf70 con 0x7f15340d0fd0 2024-01-16T14:56:40.374 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-16T14:56:40.374 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"d39fe2d6-b323-4e7f-9f15-004380a58c0d","created":"2024-01-16T14:54:27.603846+0000","modified":"2024-01-16T14:56:39.710990+0000","last_up_change":"2024-01-16T14:54:37.775916+0000","last_in_change":"2024-01-16T14:54:30.186010+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-16T14:54:39.405869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"70a24a3c-be2b-4420-855a-5740cd033b47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6809","nonce":1314027006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6811","nonce":1314027006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6815","nonce":1314027006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":1314027006},{"type":"v1","addr":"172.21.15.26:6813","nonce":1314027006}]},"public_addr":"172.21.15.26:6809/1314027006","cluster_addr":"172.21.15.26:6811/1314027006","heartbeat_back_addr":"172.21.15.26:6815/1314027006","heartbeat_front_addr":"172.21.15.26:6813/1314027006","state":["exists","up"]},{"osd":1,"uuid":"6b5503f8-087d-41d5-a7a9-c63e02c01c36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6801","nonce":795239518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6803","nonce":795239518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6807","nonce":795239518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":795239518},{"type":"v1","addr":"172.21.15.26:6805","nonce":795239518}]},"public_addr":"172.21.15.26:6801/795239518","cluster_addr":"172.21.15.26:6803/795239518","heartbeat_back_addr":"172.21.15.26:6807/795239518","heartbeat_front_addr":"172.21.15.26:6805/795239518","state":["exists","up"]},{"osd":2,"uuid":"341f4396-d5d7-4489-9f9d-16991e80028b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6825","nonce":1003733076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6827","nonce":1003733076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6831","nonce":1003733076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1003733076},{"type":"v1","addr":"172.21.15.26:6829","nonce":1003733076}]},"public_addr":"172.21.15.26:6825/1003733076","cluster_addr":"172.21.15.26:6827/1003733076","heartbeat_back_addr":"172.21.15.26:6831/1003733076","heartbeat_front_addr":"172.21.15.26:6829/1003733076","state":["exists","up"]},{"osd":3,"uuid":"b35b6724-3ca8-497d-b775-38a3d40791a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6817","nonce":600555792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6819","nonce":600555792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6823","nonce":600555792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":600555792},{"type":"v1","addr":"172.21.15.26:6821","nonce":600555792}]},"public_addr":"172.21.15.26:6817/600555792","cluster_addr":"172.21.15.26:6819/600555792","heartbeat_back_addr":"172.21.15.26:6823/600555792","heartbeat_front_addr":"172.21.15.26:6821/600555792","state":["exists","up"]},{"osd":4,"uuid":"998722f9-d9d5-4927-a70e-079f6f407ce1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6809","nonce":2738183985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6811","nonce":2738183985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6815","nonce":2738183985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":2738183985},{"type":"v1","addr":"172.21.15.93:6813","nonce":2738183985}]},"public_addr":"172.21.15.93:6809/2738183985","cluster_addr":"172.21.15.93:6811/2738183985","heartbeat_back_addr":"172.21.15.93:6815/2738183985","heartbeat_front_addr":"172.21.15.93:6813/2738183985","state":["exists","up"]},{"osd":5,"uuid":"f9b0daec-8bb4-4fe0-b123-1600aaab32cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6825","nonce":1651349923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6827","nonce":1651349923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6831","nonce":1651349923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":1651349923},{"type":"v1","addr":"172.21.15.93:6829","nonce":1651349923}]},"public_addr":"172.21.15.93:6825/1651349923","cluster_addr":"172.21.15.93:6827/1651349923","heartbeat_back_addr":"172.21.15.93:6831/1651349923","heartbeat_front_addr":"172.21.15.93:6829/1651349923","state":["exists","up"]},{"osd":6,"uuid":"512d1b6a-54bf-4854-84f6-3623eb3b4047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6801","nonce":1448583470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6803","nonce":1448583470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6807","nonce":1448583470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":1448583470},{"type":"v1","addr":"172.21.15.93:6805","nonce":1448583470}]},"public_addr":"172.21.15.93:6801/1448583470","cluster_addr":"172.21.15.93:6803/1448583470","heartbeat_back_addr":"172.21.15.93:6807/1448583470","heartbeat_front_addr":"172.21.15.93:6805/1448583470","state":["exists","up"]},{"osd":7,"uuid":"e49da02e-f545-4805-8405-5425fac2faa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6817","nonce":3713418070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6819","nonce":3713418070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6824","nonce":3713418070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":3713418070},{"type":"v1","addr":"172.21.15.93:6821","nonce":3713418070}]},"public_addr":"172.21.15.93:6817/3713418070","cluster_addr":"172.21.15.93:6819/3713418070","heartbeat_back_addr":"172.21.15.93:6824/3713418070","heartbeat_front_addr":"172.21.15.93:6821/3713418070","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.297757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.338839+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:36.756519+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.361801+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.323303+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.330610+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.357298+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-16T14:54:35.312616+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6837/406936670":"2024-01-17T14:56:36.683836+0000","172.21.15.93:6835/406936670":"2024-01-17T14:56:36.683836+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 msgr2=0x7f152006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 0x7f152006da50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1524009cd0 tx=0x7f1524009210 comp rx=0 tx=0).stop 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 msgr2=0x7f15340cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 0x7f15340cb250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1528016000 tx=0x7f1528004750 comp rx=0 tx=0).stop 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 shutdown_connections 2024-01-16T14:56:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f152006b590 0x7f152006da50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f153415e330 0x7f153414fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15340d0b70 0x7f15340b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 --2- 172.21.15.26:0/555303600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15340d0fd0 0x7f15340cb250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 >> 172.21.15.26:0/555303600 conn(0x7f15340b3800 msgr2=0x7f153406c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 shutdown_connections 2024-01-16T14:56:40.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.395+0000 7f1539102700 1 -- 172.21.15.26:0/555303600 wait complete. 2024-01-16T14:56:40.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-16T14:54:39.405869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-16T14:56:40.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-16T14:56:40.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.549+0000 7ff3af548700 1 Processor -- start 2024-01-16T14:56:40.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.549+0000 7ff3af548700 1 -- start start 2024-01-16T14:56:40.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3af548700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff3a806fe90 0x7ff3a8070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3af548700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a815f250 con 0x7ff3a806afd0 2024-01-16T14:56:40.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3af548700 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a8161cd0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3af548700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a815f070 con 0x7ff3a806fe90 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3acae3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3a806afd0 0x7ff3a815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38284/0 (socket says 172.21.15.26:38284) 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3acae3700 1 -- 172.21.15.26:0/1466301481 learned_addr learned my addr 172.21.15.26:0/1466301481 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.550+0000 7ff3ad2e4700 1 --2- 172.21.15.26:0/1466301481 >> v2:172.21.15.93:3300/0 conn(0x7ff3a806fe90 0x7ff3a8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475032771 0 0) 0x7ff3a815f250 con 0x7ff3a806afd0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3940035b0 con 0x7ff3a806afd0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff3ad2e4700 1 -- 172.21.15.26:0/1466301481 >> v1:172.21.15.93:6789/0 conn(0x7ff3a815ebe0 legacy=0x7ff3a80add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47176770 0 0) 0x7ff3a8161cd0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff3ad2e4700 1 -- 172.21.15.26:0/1466301481 >> v1:172.21.15.26:6789/0 conn(0x7ff3a806afd0 legacy=0x7ff3a815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff3ad2e4700 1 -- 172.21.15.26:0/1466301481 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8161fb0 con 0x7ff3a806fe90 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff3ad2e4700 1 --2- 172.21.15.26:0/1466301481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8070270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009840 tx=0x7ff3a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5736de4d499f4af6 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011830 con 0x7ff3a806fe90 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4011990 con 0x7ff3a806fe90 2024-01-16T14:56:40.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.551+0000 7ff39ffff700 1 -- 172.21.15.26:0/1466301481 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011d20 con 0x7ff3a806fe90 2024-01-16T14:56:40.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 -- 172.21.15.26:0/1466301481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 msgr2=0x7ff3a8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 --2- 172.21.15.26:0/1466301481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8070270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009840 tx=0x7ff3a4009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 -- 172.21.15.26:0/1466301481 shutdown_connections 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 --2- 172.21.15.26:0/1466301481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8070270 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 -- 172.21.15.26:0/1466301481 >> 172.21.15.26:0/1466301481 conn(0x7ff3a80b30c0 msgr2=0x7ff3a806c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 -- 172.21.15.26:0/1466301481 shutdown_connections 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.552+0000 7ff3af548700 1 -- 172.21.15.26:0/1466301481 wait complete. 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.553+0000 7ff3af548700 1 Processor -- start 2024-01-16T14:56:40.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.553+0000 7ff3af548700 1 -- start start 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3a806afd0 0x7ff3a813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b4370 con 0x7ff3a815ebe0 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80b4930 con 0x7ff3a806fe90 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3af548700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81f3a10 con 0x7ff3a806afd0 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3adae5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3adae5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57452/0 (socket says 172.21.15.26:57452) 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3adae5700 1 -- 172.21.15.26:0/1400115451 learned_addr learned my addr 172.21.15.26:0/1400115451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3acae3700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.554+0000 7ff3ad2e4700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3a806afd0 0x7ff3a813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 -- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3a806afd0 msgr2=0x7ff3a813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3a806afd0 0x7ff3a813ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 -- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 msgr2=0x7ff3a8141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 -- 172.21.15.26:0/1400115451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40094a0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3adae5700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c870 tx=0x7ff3a000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0003e30 con 0x7ff3a815ebe0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0005070 con 0x7ff3a815ebe0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a813d690 con 0x7ff3a815ebe0 2024-01-16T14:56:40.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0005420 con 0x7ff3a815ebe0 2024-01-16T14:56:40.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.555+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a80b45c0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.556+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a80b4ac0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.557+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7ff3a0005580 con 0x7ff3a815ebe0 2024-01-16T14:56:40.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.557+0000 7ff39dffb700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 0x7ff38c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.557+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ff3a0089300 con 0x7ff3a815ebe0 2024-01-16T14:56:40.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.557+0000 7ff3ad2e4700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 0x7ff38c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.558+0000 7ff3ad2e4700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 0x7ff38c06da50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.562+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3a00540e0 con 0x7ff3a815ebe0 2024-01-16T14:56:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.775+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3a8075480 con 0x7ff3a815ebe0 2024-01-16T14:56:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.775+0000 7ff39dffb700 1 -- 172.21.15.26:0/1400115451 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3a0017020 con 0x7ff3a815ebe0 2024-01-16T14:56:40.757 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-16T14:56:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 msgr2=0x7ff38c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 0x7ff38c06da50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a401c040 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 msgr2=0x7ff3a81422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c870 tx=0x7ff3a000cb80 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 shutdown_connections 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7ff38c06b590 0x7ff38c06da50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a815ebe0 0x7ff3a81422f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3a806afd0 0x7ff3a813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 --2- 172.21.15.26:0/1400115451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3a806fe90 0x7ff3a8141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 >> 172.21.15.26:0/1400115451 conn(0x7ff3a80b30c0 msgr2=0x7ff3a80b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 shutdown_connections 2024-01-16T14:56:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.777+0000 7ff3af548700 1 -- 172.21.15.26:0/1400115451 wait complete. 2024-01-16T14:56:40.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-16T14:56:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.930+0000 7f6faef57700 1 Processor -- start 2024-01-16T14:56:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.930+0000 7f6faef57700 1 -- start start 2024-01-16T14:56:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6faef57700 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6fa81447b0 0x7f6fa815b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6faef57700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 0x7f6fa80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6faef57700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa815bbf0 con 0x7f6fa80b4f50 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6faef57700 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa80b53e0 con 0x7f6fa81447b0 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6faef57700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa815bed0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6fad4f4700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6fa80b4f50 0x7f6fa815e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38294/0 (socket says 172.21.15.26:38294) 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.931+0000 7f6fad4f4700 1 -- 172.21.15.26:0/879000568 learned_addr learned my addr 172.21.15.26:0/879000568 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/879000568 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 0x7f6fa80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6faccf3700 1 --2- 172.21.15.26:0/879000568 >> v2:172.21.15.93:3300/0 conn(0x7f6fa81447b0 0x7f6fa815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229085106 0 0) 0x7f6fa815bbf0 con 0x7f6fa80b4f50 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f980035b0 con 0x7f6fa80b4f50 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 -- 172.21.15.26:0/879000568 >> v2:172.21.15.93:3300/0 conn(0x7f6fa81447b0 msgr2=0x7f6fa815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3140807888 0 0) 0x7f6f980035b0 con 0x7f6fa80b4f50 2024-01-16T14:56:40.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/879000568 >> v2:172.21.15.93:3300/0 conn(0x7f6fa81447b0 0x7f6fa815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 -- 172.21.15.26:0/879000568 >> v1:172.21.15.26:6789/0 conn(0x7f6fa80b4f50 legacy=0x7f6fa815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 -- 172.21.15.26:0/879000568 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa815c1b0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/879000568 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 0x7f6fa80b4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009840 tx=0x7f6f94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a642a330a8c7b540 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.932+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00019f0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94004030 con 0x7f6fa80b3c80 2024-01-16T14:56:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6fa77fe700 1 -- 172.21.15.26:0/879000568 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940043a0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 -- 172.21.15.26:0/879000568 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 msgr2=0x7f6fa80b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 --2- 172.21.15.26:0/879000568 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 0x7f6fa80b4a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009840 tx=0x7f6f94009b50 comp rx=0 tx=0).stop 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 -- 172.21.15.26:0/879000568 shutdown_connections 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 --2- 172.21.15.26:0/879000568 >> v2:172.21.15.93:3300/0 conn(0x7f6fa81447b0 0x7f6fa815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 --2- 172.21.15.26:0/879000568 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa80b3c80 0x7f6fa80b4a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 -- 172.21.15.26:0/879000568 >> 172.21.15.26:0/879000568 conn(0x7f6fa80b27d0 msgr2=0x7f6fa80b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 -- 172.21.15.26:0/879000568 shutdown_connections 2024-01-16T14:56:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.933+0000 7f6faef57700 1 -- 172.21.15.26:0/879000568 wait complete. 2024-01-16T14:56:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.935+0000 7f6faef57700 1 Processor -- start 2024-01-16T14:56:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.935+0000 7f6faef57700 1 -- start start 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fa80b4f50 0x7f6fa81389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa81447b0 0x7f6fa8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa80cfdb0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8136160 con 0x7f6fa80b4f50 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faef57700 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8136440 con 0x7f6fa81447b0 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57468/0 (socket says 172.21.15.26:57468) 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 -- 172.21.15.26:0/1894180585 learned_addr learned my addr 172.21.15.26:0/1894180585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6fad4f4700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa81447b0 0x7f6fa8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fa80b4f50 0x7f6fa81389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 -- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa81447b0 msgr2=0x7f6fa8159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa81447b0 0x7f6fa8159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 -- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fa80b4f50 msgr2=0x7f6fa81389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fa80b4f50 0x7f6fa81389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 -- 172.21.15.26:0/1894180585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f940094a0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6faccf3700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00e870 tx=0x7f6f9c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c009590 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.936+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c013680 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.937+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0139f0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.937+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa8136720 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.937+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa8136ce0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.938+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa8075480 con 0x7f6fa80b3c80 2024-01-16T14:56:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.938+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f6f9c013b50 con 0x7f6fa80b3c80 2024-01-16T14:56:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.939+0000 7f6fa57fa700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 0x7f6f8c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-16T14:56:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.939+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6f9c089bf0 con 0x7f6fa80b3c80 2024-01-16T14:56:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.939+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 0x7f6f8c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-16T14:56:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.939+0000 7f6fa7fff700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 0x7f6f8c06da50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6f940090b0 tx=0x7f6f94008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-16T14:56:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:40.944+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f9c054940 con 0x7f6fa80b3c80 2024-01-16T14:56:41.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.172+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fa80d0520 con 0x7f6fa80b3c80 2024-01-16T14:56:41.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.173+0000 7f6fa57fa700 1 -- 172.21.15.26:0/1894180585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v36) v1 ==== 80+0+120 (secure 0 0 0) 0x7f6f9c021080 con 0x7f6fa80b3c80 2024-01-16T14:56:41.155 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.93:6837/406936670 2024-01-17T14:56:36.683836+0000 2024-01-16T14:56:41.155 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.93:6835/406936670 2024-01-17T14:56:36.683836+0000 2024-01-16T14:56:41.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.179+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 msgr2=0x7f6f8c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:41.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.179+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 0x7f6f8c06da50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6f940090b0 tx=0x7f6f94008030 comp rx=0 tx=0).stop 2024-01-16T14:56:41.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.179+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 msgr2=0x7f6fa813b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-16T14:56:41.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.179+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00e870 tx=0x7f6f9c00eb80 comp rx=0 tx=0).stop 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 shutdown_connections 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6fa81447b0 0x7f6fa8159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:6832/190365378,v1:172.21.15.93:6833/190365378] conn(0x7f6f8c06b590 0x7f6f8c06da50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fa80b3c80 0x7f6fa813b960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 --2- 172.21.15.26:0/1894180585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6fa80b4f50 0x7f6fa81389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-16T14:56:41.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.181+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 >> 172.21.15.26:0/1894180585 conn(0x7f6fa80b27d0 msgr2=0x7f6fa815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-16T14:56:41.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.182+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 shutdown_connections 2024-01-16T14:56:41.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-16T14:56:41.185+0000 7f6faef57700 1 -- 172.21.15.26:0/1894180585 wait complete. 2024-01-16T14:56:41.171 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-16T14:56:41.193 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-16T14:56:41.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-16T14:56:41.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-16T14:56:53.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:56:53.106 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:57:23.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:57:23.184 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:57:53.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:57:53.280 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:23.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:23.351 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:53.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:58:53.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:23.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:23.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:53.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T14:59:53.624 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:23.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:23.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:53.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:00:53.767 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:23.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:23.830 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:41.200 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 745, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1374, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr raise socket.timeout() socket.timeout 2024-01-16T15:01:41.201 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 697, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1359, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv raise socket.timeout() socket.timeout 2024-01-16T15:01:53.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:01:53.909 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:23.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:23.990 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:54.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:02:54.087 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:24.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:24.225 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:54.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:03:54.334 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:24.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:24.429 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:54.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:04:54.501 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:24.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:24.623 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:54.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:05:54.689 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:24.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:24.752 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:54.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:06:54.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:24.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:24.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:54.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:07:54.966 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:25.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:25.038 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:55.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:08:55.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:25.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:25.223 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:55.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:09:55.293 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:25.372 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:25.376 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:55.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:10:55.448 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:25.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:25.537 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:55.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:11:55.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:25.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:25.725 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:55.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:12:55.802 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:25.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:25.885 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:55.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:13:55.958 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:26.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:26.041 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:56.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:14:56.114 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:26.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:26.261 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:56.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:15:56.343 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:26.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:26.416 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:56.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:16:56.557 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:26.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:26.697 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:56.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:17:56.852 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:27.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:27.016 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:57.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:18:57.079 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:27.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:27.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:57.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:19:57.369 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:27.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:27.499 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:57.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:20:57.589 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:27.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:27.736 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:57.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:21:57.851 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:27.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:27.998 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:58.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:22:58.083 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:28.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:28.164 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:58.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:23:58.283 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:28.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:28.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:58.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:24:58.552 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:28.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:28.642 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:58.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:25:58.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:28.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:28.895 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:58.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:26:58.976 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:29.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:29.039 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:59.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:27:59.163 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:29.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:29.312 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:59.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:28:59.379 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:29.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:29.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:59.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:29:59.619 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:29.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:29.766 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:59.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:30:59.931 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:30.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:31:30.045 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:00.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:00.151 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:30.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:32:30.281 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:00.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:00.348 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:30.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:33:30.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:00.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:00.541 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:30.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:34:30.687 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:00.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:00.758 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:30.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:35:30.916 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:00.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:00.978 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:31.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:36:31.119 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:01.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:01.182 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:31.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:37:31.322 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:01.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:01.469 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:31.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:38:31.582 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:01.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:01.729 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:31.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:39:31.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:02.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:02.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:32.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:40:32.189 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:02.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:02.319 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:32.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:41:32.509 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:02.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:02.606 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:32.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:42:32.688 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:02.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:02.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:32.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:43:32.987 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:03.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:03.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:33.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:44:33.199 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:03.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:03.292 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:33.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:45:33.427 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:03.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:03.533 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:33.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:46:33.663 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:03.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:03.819 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:33.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:47:33.890 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:03.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:03.986 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:34.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:48:34.121 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:04.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:04.246 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:34.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:49:34.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:04.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:04.481 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:34.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:50:34.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:04.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:04.675 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:34.736 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:51:34.743 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:04.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:04.820 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:34.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:52:34.977 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:05.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:05.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:35.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:53:35.195 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:05.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:05.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:35.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:54:35.416 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:05.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:05.487 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:35.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:55:35.642 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:05.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:05.796 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:35.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:56:35.944 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:06.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:06.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:36.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:57:36.213 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:06.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:06.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:36.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:58:36.565 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:06.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:06.680 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:36.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T15:59:36.759 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:06.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:06.890 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:37.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:00:37.044 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:07.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:07.174 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:37.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:01:37.303 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:07.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:07.454 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:37.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:02:37.592 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:07.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:07.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:37.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:03:37.801 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:07.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:07.948 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:38.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:04:38.088 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:08.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:08.187 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:38.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:05:38.339 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:08.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:08.412 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:38.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:06:38.551 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:08.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:08.647 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:38.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:07:38.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:08.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:08.900 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:39.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:08:39.056 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:09.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:09.186 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:39.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:09:39.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:09.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:09.455 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:39.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:10:39.628 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:09.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:09.713 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:39.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:11:39.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:09.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:09.936 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:40.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:12:40.057 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:10.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:10.126 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:40.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:13:40.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:10.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:10.406 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:40.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:14:40.537 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:10.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:10.842 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:40.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:15:40.914 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:10.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:11.002 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:41.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:16:41.158 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:11.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:11.304 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:41.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:17:41.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:11.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:11.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:41.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:18:41.662 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:11.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:11.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:41.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:19:41.907 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:12.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:12.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:42.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:20:42.158 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:12.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:12.315 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:42.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:21:42.461 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:12.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:12.600 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:42.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:22:42.683 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:12.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:12.840 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:42.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:23:42.924 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:13.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:13.005 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:43.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:24:43.089 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:13.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:13.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:43.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:25:43.292 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:13.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:13.430 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:43.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:26:43.528 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:13.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:13.650 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:43.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:27:43.764 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:13.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:13.832 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:43.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:28:43.904 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:14.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:14.065 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:44.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:29:44.145 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:14.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:14.302 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:44.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:30:44.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:14.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:14.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:44.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:31:44.652 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:14.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:14.774 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:44.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:32:44.847 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:14.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:14.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:45.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:33:45.067 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:15.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:15.128 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:45.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:34:45.261 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:15.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:15.408 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:45.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:35:45.530 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:15.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:15.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:45.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:36:45.807 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:15.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:15.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:46.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:37:46.043 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:16.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:16.207 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:46.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:38:46.355 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:16.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:16.493 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:46.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:39:46.657 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:16.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:16.806 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:46.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:40:46.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:17.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:17.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:47.210 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:41:47.213 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:17.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:17.311 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:47.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:42:47.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:17.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:17.468 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:47.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:43:47.535 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:17.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:17.605 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:47.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:44:47.693 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:17.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:17.783 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:47.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:45:47.865 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:17.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:17.929 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:47.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:46:48.000 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:18.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:18.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:48.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:47:48.127 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:18.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:18.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:48.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:48:48.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:18.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:18.545 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:48.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:49:48.617 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:18.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:18.756 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:48.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:50:48.971 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:19.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:19.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:49.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:51:49.189 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:19.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:19.303 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:49.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:52:49.443 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:19.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:19.599 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:49.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:53:49.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:19.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:19.852 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:49.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:54:50.000 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:20.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:20.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:50.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:55:50.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:20.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:20.264 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:50.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:56:50.385 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:20.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:20.523 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:50.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:57:50.661 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:20.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:20.967 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:51.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:58:51.130 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:21.474 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:21.477 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:51.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T16:59:51.633 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:21.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:21.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:51.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:00:51.862 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:21.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:21.996 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:52.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:01:52.142 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:22.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:22.281 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:52.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:02:52.347 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:22.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:22.474 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:52.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:03:52.846 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:22.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:22.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:53.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:04:53.031 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:23.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:23.152 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:53.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:05:53.282 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:23.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:23.355 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:53.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:06:53.442 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:23.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:23.622 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:53.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:07:53.744 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:23.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:23.815 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:53.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:08:53.887 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:24.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:24.034 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:54.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:09:54.199 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:24.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:24.276 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:54.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:10:54.374 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:24.502 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:24.505 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:54.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:11:54.601 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:24.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:24.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:54.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:12:54.820 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:24.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:24.889 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:55.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:13:55.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:25.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:25.186 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:55.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:14:55.300 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:25.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:25.405 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:55.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:15:55.500 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:25.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:25.589 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:55.708 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:16:55.712 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:25.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:25.851 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:56.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:17:56.014 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:26.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:26.228 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:56.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:18:56.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:26.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:26.458 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:56.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:19:56.579 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:26.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:26.700 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:56.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:20:56.815 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:26.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:26.930 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:57.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:21:57.060 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:27.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:27.173 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:57.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:22:57.329 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:27.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:27.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:57.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:23:57.540 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:27.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:27.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:57.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:24:57.799 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:27.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:27.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:58.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:25:58.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:28.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:28.175 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:58.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:26:58.254 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:28.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:28.385 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:58.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:27:58.444 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:28.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:28.504 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:58.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:28:58.646 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:28.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:28.777 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:58.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:29:58.918 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:29.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:29.003 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:59.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:30:59.073 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:29.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:29.216 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:59.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:31:59.371 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:29.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:29.501 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:59.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:32:59.608 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:29.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:29.724 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:59.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:33:59.838 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:29.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:34:29.970 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:00.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:00.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:30.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:35:30.157 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:00.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:00.282 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:30.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:36:30.439 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:00.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:00.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:30.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:37:30.660 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:01.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:01.017 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:31.128 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:38:31.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:01.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:01.281 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:31.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:39:31.437 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:01.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:01.595 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:31.706 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:40:31.709 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:01.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:01.781 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:31.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:41:31.853 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:01.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:02.001 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:32.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:42:32.132 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:02.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:02.287 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:32.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:43:32.427 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:02.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:02.558 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:32.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:44:32.680 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:02.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:02.836 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:32.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:45:32.911 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:02.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:02.983 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:33.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:46:33.120 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:03.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:03.246 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:33.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:47:33.316 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:03.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:03.455 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:33.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:48:33.575 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:03.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:03.706 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:33.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:49:33.770 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:03.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:03.885 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:34.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:50:34.024 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:04.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:04.188 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:34.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:51:34.265 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:04.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:04.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:34.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:52:34.546 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:04.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:04.619 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:34.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:53:34.732 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:04.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:04.889 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:34.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:54:34.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:05.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:05.125 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:35.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:55:35.289 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:05.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:05.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:35.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:56:35.559 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:05.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:05.708 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:35.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:57:35.781 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:06.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:06.119 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:36.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:58:36.206 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:06.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:06.353 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:36.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T17:59:36.419 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:06.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:06.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:36.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:00:36.739 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:06.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:06.886 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:36.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:01:36.975 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:07.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:07.097 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:37.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:02:37.219 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:07.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:07.359 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:37.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:03:37.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:07.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:07.570 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:37.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:04:37.717 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:07.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:07.955 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:38.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:05:38.028 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:08.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:08.158 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:38.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:06:38.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:08.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:08.378 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:38.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:07:38.456 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:08.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:08.609 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:38.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:08:38.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:08.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:08.807 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:38.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:09:38.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:09.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:09.088 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:39.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:10:39.233 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:09.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:09.305 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:39.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:11:39.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:09.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:09.540 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:39.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:12:39.687 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:09.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:09.750 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:40.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:13:40.025 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:10.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:10.105 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:40.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:14:40.194 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:10.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:10.301 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:40.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:15:40.457 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:10.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:10.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:40.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:16:40.735 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:10.896 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:10.899 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:41.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:17:41.138 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:11.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:11.203 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:41.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:18:41.359 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:11.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:11.497 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:41.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:19:41.586 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:11.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:11.742 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:41.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:20:41.888 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:11.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:11.985 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:42.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:21:42.150 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:12.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:12.296 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:42.359 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:22:42.363 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:12.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:12.508 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:42.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:23:42.655 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:12.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:12.802 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:42.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:24:42.875 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:12.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:12.962 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:43.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:25:43.077 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:13.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:13.140 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:43.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:26:43.264 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:13.407 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:13.412 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:43.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:27:43.501 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:13.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:13.666 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:43.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:28:43.745 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:13.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:13.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:43.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:29:43.983 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:14.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:14.137 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:44.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:30:44.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:14.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:14.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:44.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:31:44.523 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:14.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:14.663 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:44.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:32:44.741 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:14.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:14.817 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:44.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:33:44.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:15.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:15.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:45.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:34:45.191 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:15.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:15.315 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:45.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:35:45.469 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:15.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:15.792 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:45.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:36:45.913 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:16.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:16.069 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:46.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:37:46.144 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:16.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:16.212 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:46.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:38:46.285 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:16.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:16.356 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:46.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:39:46.430 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:16.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:16.518 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:46.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:40:46.582 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:16.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:16.704 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:46.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:41:46.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:17.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:17.032 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:47.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:42:47.180 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:17.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:17.252 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:47.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:43:47.366 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:17.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:17.488 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:47.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:44:47.635 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:17.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:17.775 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:47.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:45:47.888 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:18.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:18.043 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:48.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:46:48.181 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:18.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:18.328 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:48.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:47:48.408 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:18.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:18.491 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:48.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:48:48.569 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:18.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:18.657 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:48.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:49:48.776 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:18.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:18.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:49.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:50:49.046 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:19.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:19.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:49.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:51:49.282 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:19.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:19.446 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:49.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:52:49.519 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:19.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:19.590 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:49.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:53:49.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:19.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:19.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:50.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:54:50.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:20.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:20.181 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:50.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:55:50.313 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:20.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:20.458 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:50.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:56:50.742 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:20.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:20.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:50.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:57:50.896 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:21.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:21.233 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:51.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:58:51.303 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:21.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:21.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:51.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T18:59:51.573 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:21.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:21.730 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:51.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:00:51.864 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:22.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:22.084 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:52.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:01:52.162 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:22.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:22.288 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:52.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:02:52.440 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:22.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:22.578 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:52.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:03:52.704 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:22.852 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:22.855 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:52.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:04:52.942 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:23.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:23.074 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:53.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:05:53.217 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:23.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:23.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:53.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:06:53.543 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:23.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:23.689 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:53.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:07:53.828 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:23.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:23.961 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:54.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:08:54.199 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:24.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:24.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:54.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:09:54.419 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:24.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:24.498 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:54.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:10:54.594 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:24.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:24.737 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:54.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:11:54.868 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:25.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:25.013 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:55.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:12:55.287 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:25.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:25.352 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:55.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:13:55.482 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:25.583 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:25.587 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:55.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:14:55.696 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:25.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:25.847 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:55.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:15:55.994 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:26.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:26.133 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:56.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:16:56.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:26.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:26.300 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:56.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:17:56.433 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:26.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:26.571 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:56.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:18:56.641 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:26.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:26.765 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:56.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:19:56.914 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:27.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:27.059 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:57.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:20:57.190 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:27.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:27.304 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:57.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:21:57.443 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:27.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:27.509 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:57.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:22:57.646 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:27.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:27.785 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:57.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:23:57.861 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:27.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:27.980 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:58.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:24:58.110 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:28.287 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:28.290 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:58.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:25:58.445 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:28.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:28.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:58.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:26:58.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:28.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:28.815 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:58.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:27:58.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:29.139 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:29.142 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:59.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:28:59.273 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:29.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:29.336 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:59.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:29:59.406 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:29.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:29.524 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:59.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:30:59.638 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:29.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:29.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:59.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:31:59.958 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:30.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:32:30.072 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:00.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:00.147 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:30.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:33:30.260 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:00.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:00.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:30.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:34:30.479 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:00.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:00.617 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:30.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:35:30.782 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:00.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:00.938 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:31.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:36:31.069 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:01.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:01.140 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:31.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:37:31.270 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:01.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:01.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:31.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:38:31.491 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:01.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:01.645 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:31.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:39:31.792 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:01.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:01.931 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:32.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:40:32.004 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:02.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:02.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:32.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:41:32.299 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:02.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:02.397 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:32.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:42:32.459 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:02.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:02.531 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:32.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:43:32.678 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:02.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:02.809 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:32.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:44:32.923 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:03.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:03.046 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:33.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:45:33.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:03.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:03.257 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:33.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:46:33.412 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:03.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:03.518 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:33.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:47:33.631 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:03.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:03.770 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:33.913 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:48:33.917 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:04.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:04.063 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:34.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:49:34.126 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:04.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:04.193 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:34.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:50:34.278 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:04.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:04.418 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:34.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:51:34.483 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:04.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:04.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:34.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:52:34.707 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:04.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:04.786 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:34.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:53:34.943 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:05.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:05.084 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:35.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:54:35.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:05.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:05.383 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:35.469 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:55:35.473 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:05.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:05.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:35.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:56:35.674 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:05.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:05.804 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:35.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:57:35.944 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:06.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:06.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:36.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:58:36.191 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:06.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:06.342 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:36.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T19:59:36.429 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:06.502 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:06.506 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:36.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:00:36.674 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:06.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:06.746 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:36.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:01:36.867 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:06.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:06.997 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:37.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:02:37.154 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:07.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:07.301 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:37.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:03:37.363 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:07.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:07.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:37.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:04:37.577 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:07.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:07.712 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:37.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:05:37.809 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:07.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:07.941 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:38.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:06:38.081 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:08.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:08.233 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:38.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:07:38.298 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:08.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:08.369 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:38.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:08:38.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:08.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:08.580 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:38.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:09:38.669 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:08.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:08.792 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:38.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:10:38.939 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:09.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:09.069 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:39.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:11:39.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:09.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:09.489 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:39.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:12:39.585 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:09.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:09.709 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:39.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:13:39.823 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:09.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:09.954 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:40.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:14:40.110 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:10.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:10.363 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:40.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:15:40.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:10.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:10.518 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:40.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:16:40.660 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:10.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:10.799 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:40.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:17:40.867 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:11.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:11.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:41.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:18:41.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:11.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:11.413 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:41.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:19:41.700 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:11.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:11.771 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:41.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:20:41.854 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:12.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:12.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:42.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:21:42.197 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:12.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:12.346 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:42.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:22:42.408 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:12.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:12.566 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:42.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:23:42.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:12.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:12.794 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:42.929 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:24:42.932 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:13.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:13.097 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:43.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:25:43.226 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:13.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:13.348 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:43.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:26:43.479 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:13.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:13.601 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:43.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:27:43.673 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:13.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:13.748 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:43.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:28:43.820 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:13.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:13.958 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:44.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:29:44.024 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:14.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:14.160 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:44.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:30:44.300 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:14.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:14.427 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:44.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:31:44.526 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:14.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:14.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:44.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:32:44.753 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:14.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:14.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:44.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:33:44.971 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:15.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:15.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:45.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:34:45.232 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:15.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:15.390 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:45.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:35:45.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:15.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:15.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:45.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:36:45.762 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:15.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:15.894 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:45.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:37:45.973 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:16.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:16.236 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:46.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:38:46.384 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:16.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:16.531 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:46.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:39:46.661 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:16.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:16.791 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:46.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:40:46.864 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:17.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:17.012 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:47.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:41:47.141 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:17.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:17.272 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:47.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:42:47.419 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:17.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:17.549 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:47.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:43:47.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:17.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:17.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:47.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:44:47.945 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:18.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:18.092 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:48.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:45:48.261 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:18.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:18.407 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:48.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:46:48.547 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:18.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:18.625 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:48.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:47:48.732 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:18.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:18.881 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:49.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:48:49.010 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:19.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:19.188 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:49.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:49:49.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:19.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:19.428 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:49.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:50:49.533 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:19.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:19.674 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:49.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:51:49.899 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:19.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:19.979 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:50.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:52:50.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:20.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:20.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:50.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:53:50.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:20.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:20.502 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:50.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:54:50.641 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:20.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:20.755 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:50.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:55:50.844 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:20.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:20.996 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:51.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:56:51.147 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:21.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:21.279 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:51.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:57:51.400 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:21.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:21.546 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:51.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:58:51.694 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:21.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:21.772 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:51.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T20:59:51.920 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:22.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:22.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:52.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:00:52.165 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:22.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:22.330 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:52.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:01:52.466 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:22.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:22.715 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:52.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:22.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:22.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:53.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:03:53.090 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:23.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:23.235 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:53.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:04:53.394 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:23.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:23.470 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:53.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:05:53.594 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:23.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:23.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:53.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:06:53.964 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:24.055 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:24.059 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:54.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:07:54.206 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:24.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:24.354 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:54.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:08:54.434 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:24.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:24.539 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:54.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:09:54.678 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:24.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:24.800 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:54.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:10:54.954 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:25.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:25.069 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:55.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:11:55.216 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:25.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:25.371 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:55.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:12:55.476 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:25.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:25.573 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:55.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:13:55.736 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:25.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:25.869 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:55.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:14:55.940 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:26.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:26.014 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:56.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:15:56.100 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:26.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:26.246 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:56.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:16:56.403 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:26.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:26.524 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:56.658 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:17:56.662 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:26.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:26.809 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:56.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:18:56.959 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:27.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:27.098 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:57.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:19:57.167 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:27.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:27.303 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:57.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:20:57.386 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:27.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:27.533 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:57.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:21:57.689 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:27.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:27.851 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:57.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:22:57.949 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:28.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:28.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:58.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:23:58.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:28.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:28.350 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:58.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:24:58.471 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:28.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:28.535 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:58.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:25:58.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:28.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:28.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:58.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:26:58.992 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:29.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:29.139 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:59.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:27:59.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:29.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:29.275 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:59.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:28:59.433 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:29.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:29.522 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:59.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:29:59.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:29.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:29.752 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:59.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:30:59.915 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:30.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:31:30.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:00.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:00.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:30.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:32:30.333 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:00.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:00.405 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:30.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:33:30.537 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:00.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:00.609 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:30.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:34:30.731 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:00.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:00.862 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:31.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:35:31.019 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:01.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:01.093 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:31.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:36:31.171 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:01.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:01.311 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:31.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:37:31.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:01.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:01.525 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:31.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:38:31.601 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:01.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:01.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:31.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:39:31.795 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:01.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:01.923 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:32.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:40:32.080 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:02.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:02.227 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:32.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:41:32.308 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:02.469 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:02.473 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:32.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:42:32.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:02.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:02.725 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:32.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:43:32.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:02.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:02.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:33.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:44:33.100 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:03.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:03.254 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:33.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:45:33.395 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:03.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:03.549 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:33.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:46:33.639 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:03.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:03.701 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:33.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:47:33.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:04.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:04.021 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:34.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:48:34.168 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:04.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:04.299 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:34.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:49:34.437 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:04.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:04.576 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:34.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:50:34.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:04.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:04.754 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:34.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:51:34.901 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:05.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:05.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:35.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:52:35.171 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:05.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:05.238 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:35.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:53:35.383 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:05.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:05.487 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:35.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:54:35.626 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:05.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:05.849 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:35.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:55:35.949 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:06.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:06.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:36.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:56:36.214 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:06.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:06.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:36.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:57:36.451 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:06.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:06.574 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:36.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:58:36.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:06.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:06.911 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:36.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T21:59:36.989 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:07.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:07.065 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:37.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:00:37.196 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:07.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:07.345 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:37.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:01:37.445 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:07.592 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:07.596 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:37.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:02:37.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:08.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:08.064 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:38.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:03:38.169 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:08.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:08.232 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:38.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:04:38.371 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:08.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:08.528 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:38.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:05:38.658 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:08.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:08.738 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:38.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:06:38.886 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:09.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:09.014 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:39.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:07:39.089 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:09.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:09.233 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:39.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:08:39.365 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:09.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:09.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:39.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:09:39.582 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:09.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:09.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:39.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:10:39.800 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:09.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:09.883 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:39.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:11:39.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:10.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:10.093 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:40.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:12:40.170 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:10.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:10.237 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:40.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:13:40.393 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:10.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:10.515 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:40.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:14:40.655 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:10.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:10.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:40.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:15:40.864 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:11.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:11.028 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:41.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:16:41.166 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:11.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:11.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:41.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:17:41.444 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:11.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:11.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:41.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:18:41.706 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:11.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:11.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:41.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:19:41.933 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:12.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:12.056 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:42.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:20:42.161 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:12.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:12.323 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:42.401 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:21:42.405 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:12.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:12.527 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:42.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:22:42.657 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:12.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:12.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:42.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:23:42.895 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:12.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:12.975 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:43.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:24:43.131 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:13.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:13.202 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:43.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:25:43.333 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:13.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:13.446 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:43.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:26:43.552 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:13.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:13.632 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:43.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:27:43.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:13.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:13.919 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:44.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:28:44.068 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:14.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:14.188 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:44.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:29:44.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:14.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:14.474 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:44.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:30:44.542 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:14.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:14.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:44.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:31:44.693 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:14.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:14.781 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:44.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:32:44.929 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:14.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:14.994 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:45.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:33:45.150 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:15.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:15.256 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:45.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:34:45.331 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:15.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:15.395 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:45.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:35:45.540 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:15.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:15.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:45.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:36:45.853 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:15.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:15.950 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:46.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:37:46.091 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:16.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:16.228 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:46.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:38:46.304 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:16.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:16.448 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:46.592 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:39:46.596 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:16.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:16.743 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:46.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:40:46.873 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:17.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:17.005 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:47.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:41:47.143 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:17.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:17.299 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:47.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:42:47.456 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:17.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:17.610 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:47.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:43:47.742 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:17.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:17.881 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:48.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:44:48.036 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:18.088 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:18.092 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:48.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:45:48.230 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:18.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:18.312 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:48.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:46:48.459 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:18.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:18.523 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:48.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:47:48.654 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:18.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:18.792 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:48.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:48:48.940 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:19.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:19.104 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:49.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:49:49.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:19.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:19.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:49.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:50:49.553 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:19.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:19.708 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:49.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:51:49.850 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:19.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:19.913 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:50.116 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:52:50.119 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:20.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:20.224 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:50.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:53:50.304 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:20.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:20.443 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:50.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:54:50.532 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:20.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:20.662 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:50.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:55:50.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:20.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:20.931 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:51.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:56:51.079 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:21.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:21.409 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:51.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:57:51.589 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:21.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:21.737 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:51.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:58:51.867 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:22.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:22.023 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:52.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T22:59:52.171 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:22.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:22.301 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:52.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:00:52.418 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:22.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:22.571 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:52.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:01:52.644 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:22.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:22.706 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:52.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:02:52.862 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:23.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:23.026 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:53.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:03:53.139 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:23.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:23.287 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:53.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:04:53.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:23.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:23.581 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:53.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:05:53.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:23.781 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:23.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:53.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:06:53.914 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:24.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:24.036 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:54.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:07:54.176 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:24.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:24.298 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:54.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:08:54.445 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:24.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:24.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:54.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:09:54.654 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:24.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:24.738 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:54.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:10:54.878 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:25.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:25.009 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:55.180 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:11:55.184 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:25.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:25.337 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:55.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:12:55.427 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:25.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:25.507 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:55.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:13:55.580 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:25.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:25.709 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:55.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:14:55.850 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:25.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:25.938 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:56.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:15:56.086 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:26.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:26.243 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:56.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:16:56.391 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:26.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:26.456 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:56.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:17:56.535 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:26.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:26.682 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:56.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:18:56.765 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:26.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:26.868 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:56.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:19:56.991 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:27.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:27.133 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:57.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:20:57.276 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:27.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:27.449 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:57.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:21:57.520 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:27.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:27.585 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:57.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:22:57.714 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:27.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:27.839 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:57.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:23:57.963 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:28.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:28.115 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:58.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:24:58.228 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:28.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:28.360 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:58.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:25:58.657 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:28.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:28.731 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:58.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:26:58.834 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:28.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:28.957 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:59.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:27:59.112 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:29.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:29.260 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:59.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:28:59.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:29.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:29.521 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:59.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:29:59.652 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:29.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:29.791 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:59.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:30:59.865 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:29.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:31:29.971 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:00.055 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:00.059 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:30.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:32:30.214 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:00.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:00.304 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:30.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:33:30.383 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:00.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:00.450 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:30.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:34:30.574 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:00.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:00.663 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:30.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:35:30.793 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:00.913 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:00.917 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:31.069 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:36:31.073 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:01.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:01.187 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:31.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:37:31.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:01.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:01.399 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:31.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:38:31.529 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:01.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:01.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:31.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:39:31.867 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:01.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:01.956 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:32.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:40:32.018 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:02.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:02.099 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:32.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:41:32.248 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:02.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:02.387 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:32.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:42:32.450 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:02.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:02.589 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:32.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:43:32.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:02.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:02.858 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:32.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:44:32.981 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:03.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:03.055 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:33.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:45:33.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:03.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:03.197 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:33.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:46:33.285 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:03.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:03.375 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:33.469 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:47:33.473 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:03.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:03.628 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:33.688 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:48:33.691 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:03.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:03.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:33.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:49:33.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:04.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:04.083 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:34.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:50:34.247 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:04.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:04.369 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:34.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:51:34.524 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:04.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:04.654 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:34.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:52:34.793 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:04.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:04.927 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:35.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:53:35.012 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:05.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:05.176 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:35.245 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:54:35.248 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:05.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:05.379 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:35.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:55:35.501 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:05.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:05.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:35.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:56:35.746 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:05.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:05.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:35.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:57:35.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:06.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:06.172 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:36.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:58:36.260 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:06.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:06.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:36.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-16T23:59:36.580 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:06.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:06.736 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:36.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:00:36.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:06.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:06.965 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:37.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:01:37.052 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:07.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:07.134 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:37.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:02:37.223 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:07.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:07.382 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:37.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:03:37.441 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:07.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:07.578 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:37.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:04:37.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:07.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:07.878 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:38.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:05:38.025 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:08.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:08.102 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:38.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:06:38.261 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:08.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:08.359 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:38.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:07:38.464 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:08.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:08.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:38.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:08:38.686 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:08.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:08.832 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:38.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:09:38.978 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:09.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:09.045 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:39.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:10:39.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:09.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:09.339 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:39.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:11:39.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:09.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:09.534 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:39.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:12:39.687 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:09.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:09.765 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:39.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:13:39.917 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:10.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:10.055 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:40.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:14:40.121 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:10.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:10.201 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:40.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:15:40.289 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:10.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:10.369 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:40.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:16:40.457 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:10.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:10.537 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:40.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:17:40.609 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:10.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:10.682 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:40.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:18:40.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:10.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:10.910 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:41.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:19:41.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:11.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:11.106 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:41.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:20:41.212 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:11.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:11.301 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:41.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:21:41.389 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:11.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:11.512 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:41.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:22:41.642 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:11.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:11.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:41.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:23:41.935 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:12.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:12.066 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:42.175 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:24:42.179 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:12.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:12.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:42.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:25:42.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:12.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:12.537 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:42.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:26:42.683 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:12.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:12.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:42.957 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:27:42.961 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:13.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:13.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:43.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:28:43.247 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:13.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:13.397 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:43.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:29:43.533 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:13.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:13.606 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:43.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:30:43.743 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:13.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:13.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:43.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:31:43.963 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:14.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:14.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:44.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:32:44.266 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:14.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:14.347 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:44.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:33:44.477 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:14.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:14.582 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:44.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:34:44.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:14.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:14.870 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:45.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:35:45.025 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:15.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:15.103 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:45.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:36:45.185 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:15.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:15.291 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:45.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:37:45.429 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:15.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:15.507 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:45.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:38:45.591 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:15.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:15.696 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:45.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:39:45.827 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:15.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:15.939 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:46.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:40:46.079 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:16.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:16.153 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:46.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:41:46.331 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:16.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:16.436 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:46.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:42:46.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:16.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:16.697 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:46.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:43:46.761 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:16.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:16.908 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:47.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:44:47.056 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:17.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:17.136 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:47.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:45:47.199 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:17.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:17.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:47.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:46:47.452 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:17.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:17.574 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:47.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:47:47.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:17.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:17.877 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:47.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:48:47.990 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:18.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:18.129 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:48.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:49:48.240 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:18.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:18.363 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:48.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:50:48.497 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:18.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:18.619 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:48.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:51:48.758 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:18.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:18.834 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:48.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:52:48.914 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:19.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:19.027 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:49.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:53:49.175 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:19.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:19.313 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:49.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:54:49.461 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:19.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:19.552 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:49.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:55:49.675 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:19.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:19.831 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:49.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:56:49.963 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:20.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:20.096 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:50.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:57:50.168 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:20.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:20.242 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:50.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:58:50.381 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:20.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:20.521 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:50.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T00:59:50.610 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:20.675 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:20.679 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:50.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:00:50.806 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:20.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:20.882 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:51.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:01:51.038 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:21.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:21.192 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:51.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:02:51.323 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:21.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:21.438 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:51.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:03:51.776 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:21.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:21.838 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:51.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:04:51.967 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:22.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:22.101 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:52.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:05:52.240 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:22.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:22.421 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:52.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:06:52.579 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:22.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:22.673 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:52.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:07:52.811 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:22.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:22.934 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:52.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:08:53.000 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:23.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:23.145 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:53.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:09:53.310 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:23.370 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:23.374 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:53.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:10:53.494 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:23.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:23.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:53.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:11:53.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:23.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:23.886 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:53.938 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:12:53.941 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:24.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:24.022 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:54.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:13:54.162 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:24.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:24.310 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:54.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:14:54.448 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:24.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:24.588 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:54.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:15:54.734 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:24.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:24.800 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:55.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:16:55.180 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:25.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:25.252 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:55.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:17:55.408 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:25.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:25.480 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:55.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:18:55.611 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:25.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:25.759 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:55.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:19:55.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:26.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:26.054 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:56.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:20:56.127 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:26.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:26.276 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:56.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:21:56.354 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:26.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:26.428 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:56.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:22:56.492 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:26.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:26.569 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:56.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:23:56.646 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:26.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:26.719 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:56.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:24:56.956 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:27.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:27.032 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:57.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:25:57.201 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:27.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:27.348 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:57.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:26:57.453 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:27.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:27.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:57.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:27:57.757 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:27.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:27.887 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:58.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:28:58.018 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:28.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:28.141 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:58.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:29:58.204 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:28.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:28.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:58.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:30:58.434 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:28.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:28.508 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:58.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:31:58.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:28.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:28.733 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:58.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:32:58.872 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:28.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:28.969 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:59.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:33:59.099 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:29.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:29.239 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:59.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:34:59.311 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:29.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:29.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:59.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:35:59.558 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:29.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:29.696 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:59.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:36:59.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:29.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:29.902 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:59.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:37:59.966 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:30.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:38:30.108 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:00.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:00.256 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:30.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:39:30.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:00.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:00.543 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:30.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:40:30.631 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:00.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:00.788 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:30.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:41:30.919 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:01.021 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:01.024 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:31.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:42:31.163 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:01.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:01.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:31.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:43:31.386 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:01.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:01.540 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:31.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:44:31.671 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:01.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:01.819 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:31.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:45:31.950 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:02.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:02.106 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:32.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:46:32.169 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:02.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:02.328 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:32.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:47:32.406 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:02.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:02.545 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:32.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:48:32.684 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:02.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:02.815 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:32.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:49:32.955 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:03.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:03.018 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:33.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:50:33.166 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:03.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:03.230 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:33.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:51:33.361 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:03.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:03.442 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:33.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:52:33.526 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:03.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:03.588 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:33.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:53:33.660 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:03.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:03.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:33.810 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:54:33.814 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:03.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:03.875 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:33.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:55:33.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:04.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:04.117 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:34.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:56:34.198 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:04.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:04.277 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:34.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:57:34.354 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:04.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:04.428 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:34.498 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:58:34.501 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:04.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:04.572 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:34.640 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T01:59:34.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:04.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:04.790 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:34.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:00:34.945 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:05.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:05.068 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:35.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:01:35.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:05.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:05.395 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:35.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:02:35.544 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:05.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:05.615 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:35.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:03:35.713 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:05.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:05.861 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:35.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:04:35.983 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:06.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:06.131 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:36.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:05:36.197 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:06.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:06.260 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:36.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:06:36.322 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:06.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:06.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:36.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:07:36.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:06.712 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:06.716 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:36.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:08:36.794 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:06.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:06.953 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:37.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:09:37.083 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:07.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:07.232 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:37.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:10:37.396 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:07.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:07.460 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:37.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:11:37.595 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:07.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:07.670 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:37.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:12:37.801 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:07.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:07.973 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:38.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:13:38.112 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:08.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:08.251 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:38.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:14:38.366 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:08.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:08.488 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:38.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:15:38.618 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:08.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:08.732 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:38.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:16:38.878 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:09.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:09.009 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:39.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:17:39.157 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:09.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:09.313 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:39.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:18:40.670 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:10.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:10.850 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:40.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:19:40.996 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:11.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:11.228 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:41.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:20:41.392 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:11.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:11.458 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:41.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:21:41.569 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:11.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:11.709 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:41.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:22:41.874 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:11.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:11.996 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:42.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:23:42.134 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:12.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:12.289 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:42.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:24:42.403 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:12.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:12.509 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:42.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:25:42.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:12.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:12.795 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:42.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:26:42.865 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:13.019 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:13.023 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:43.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:27:43.161 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:13.273 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:13.277 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:43.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:28:43.365 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:13.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:13.429 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:43.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:29:43.516 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:13.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:13.587 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:43.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:30:43.696 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:13.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:13.835 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:43.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:31:43.932 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:14.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:14.104 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:44.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:32:44.267 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:14.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:14.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:44.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:33:44.548 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:14.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:14.687 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:44.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:34:44.760 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:14.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:14.846 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:44.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:35:44.926 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:15.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:15.042 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:45.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:36:45.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:15.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:15.199 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:45.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:37:45.278 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:15.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:15.374 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:45.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:38:45.446 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:15.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:15.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:45.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:39:45.598 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:15.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:15.673 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:45.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:40:45.738 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:15.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:15.821 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:45.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:41:45.900 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:15.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:15.973 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:46.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:42:46.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:16.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:16.129 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:46.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:43:46.205 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-17T02:44:05.048 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-01-17T02:44:05.048 DEBUG:teuthology.task.console_log:Killing console logger for smithi026 2024-01-17T02:44:05.049 DEBUG:teuthology.task.console_log:Killing console logger for smithi093 2024-01-17T02:44:05.049 DEBUG:teuthology.exit:Finished running handlers